Morten Hjorth-Jensen
2134 lines
108 KiB
Plaintext
2134 lines
108 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "43efcac0",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
|
||
"doconce format html linalg.do.txt -->"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "074ac7c2",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"# Linear Algebra, Handling of Arrays and more Python Features"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0be7d59e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Introduction\n",
|
||
"\n",
|
||
"The aim of this set of lectures is to review some central linear algebra algorithms that we will need in our \n",
|
||
"data analysis part and in the construction of Machine Learning algorithms (ML). \n",
|
||
"This will allow us to introduce some central programming features of high-level languages like Python and \n",
|
||
"compiled languages like C++ and/or Fortran. \n",
|
||
"\n",
|
||
"As discussed in the introductory notes, these series of lectures focuses both on using\n",
|
||
"central Python packages like **tensorflow** and **scikit-learn** as well\n",
|
||
"as writing your own codes for some central ML algorithms. The\n",
|
||
"latter can be written in a language of your choice, be it Python, Julia, R,\n",
|
||
"Rust, C++, Fortran etc. In order to avoid confusion however, in these lectures we will limit our\n",
|
||
"attention to Python, C++ and Fortran."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5e77e6c1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Important Matrix and vector handling packages\n",
|
||
"\n",
|
||
"There are several central software packages for linear algebra and eigenvalue problems. Several of the more\n",
|
||
"popular ones have been wrapped into ofter software packages like those from the widely used text **Numerical Recipes**. The original source codes in many of the available packages are often taken from the widely used\n",
|
||
"software package LAPACK, which follows two other popular packages\n",
|
||
"developed in the 1970s, namely EISPACK and LINPACK. We describe them shortly here.\n",
|
||
"\n",
|
||
" * LINPACK: package for linear equations and least square problems.\n",
|
||
"\n",
|
||
" * LAPACK:package for solving symmetric, unsymmetric and generalized eigenvalue problems. From LAPACK's website <http://www.netlib.org> it is possible to download for free all source codes from this library. Both C/C++ and Fortran versions are available.\n",
|
||
"\n",
|
||
" * BLAS (I, II and III): (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations. Blas I is vector operations, II vector-matrix operations and III matrix-matrix operations. Highly parallelized and efficient codes, all available for download from <http://www.netlib.org>.\n",
|
||
"\n",
|
||
"When dealing with matrices and vectors a central issue is memory\n",
|
||
"handling and allocation. If our code is written in Python the way we\n",
|
||
"declare these objects and the way they are handled, interpreted and\n",
|
||
"used by say a linear algebra library, requires codes that interface\n",
|
||
"our Python program with such libraries. For Python programmers,\n",
|
||
"**Numpy** is by now the standard Python package for numerical arrays in\n",
|
||
"Python as well as the source of functions which act on these\n",
|
||
"arrays. These functions span from eigenvalue solvers to functions that\n",
|
||
"compute the mean value, variance or the covariance matrix. If you are\n",
|
||
"not familiar with how arrays are handled in say Python or compiled\n",
|
||
"languages like C++ and Fortran, the sections in this chapter may be\n",
|
||
"useful. For C++ programmer, **Armadillo** is widely used library for\n",
|
||
"linear algebra and eigenvalue problems. In addition it offers a\n",
|
||
"convenient way to handle and organize arrays. We discuss this library\n",
|
||
"as well. Before we proceed we believe it may be convenient to repeat some basic features of \n",
|
||
" matrices and vectors."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2eed76da",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Basic Matrix Features\n",
|
||
"\n",
|
||
"Matrix properties reminder"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "109ff37c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A} =\n",
|
||
" \\begin{bmatrix} a_{11} & a_{12} & a_{13} & a_{14} \\\\\n",
|
||
" a_{21} & a_{22} & a_{23} & a_{24} \\\\\n",
|
||
" a_{31} & a_{32} & a_{33} & a_{34} \\\\\n",
|
||
" a_{41} & a_{42} & a_{43} & a_{44}\n",
|
||
" \\end{bmatrix}\\qquad\n",
|
||
"\\mathbf{I} =\n",
|
||
" \\begin{bmatrix} 1 & 0 & 0 & 0 \\\\\n",
|
||
" 0 & 1 & 0 & 0 \\\\\n",
|
||
" 0 & 0 & 1 & 0 \\\\\n",
|
||
" 0 & 0 & 0 & 1\n",
|
||
" \\end{bmatrix}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c87c75d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The inverse of a matrix is defined by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "aab763d1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}^{-1} \\cdot \\mathbf{A} = I\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f382e0f8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<table class=\"dotable\" border=\"1\">\n",
|
||
"<thead>\n",
|
||
"<tr><th align=\"center\"> Relations </th> <th align=\"center\"> Name </th> <th align=\"center\"> matrix elements </th> </tr>\n",
|
||
"</thead>\n",
|
||
"<tbody>\n",
|
||
"<tr><td align=\"center\"> $A = A^{T}$ </td> <td align=\"center\"> symmetric </td> <td align=\"center\"> $a_{ij} = a_{ji}$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = \\left (A^{T} \\right )^{-1}$ </td> <td align=\"center\"> real orthogonal </td> <td align=\"center\"> $\\sum_k a_{ik} a_{jk} = \\sum_k a_{ki} a_{kj} = \\delta_{ij}$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = A^{ * }$ </td> <td align=\"center\"> real matrix </td> <td align=\"center\"> $a_{ij} = a_{ij}^{ * }$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = A^{\\dagger}$ </td> <td align=\"center\"> hermitian </td> <td align=\"center\"> $a_{ij} = a_{ji}^{ * }$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = \\left (A^{\\dagger} \\right )^{-1}$ </td> <td align=\"center\"> unitary </td> <td align=\"center\"> $\\sum_k a_{ik} a_{jk}^{ * } = \\sum_k a_{ki}^{ * } a_{kj} = \\delta_{ij}$ </td> </tr>\n",
|
||
"</tbody>\n",
|
||
"</table>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5d394b4e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"### Some famous Matrices\n",
|
||
"\n",
|
||
" * Diagonal if $a_{ij}=0$ for $i\\ne j$\n",
|
||
"\n",
|
||
" * Upper triangular if $a_{ij}=0$ for $i > j$\n",
|
||
"\n",
|
||
" * Lower triangular if $a_{ij}=0$ for $i < j$\n",
|
||
"\n",
|
||
" * Upper Hessenberg if $a_{ij}=0$ for $i > j+1$\n",
|
||
"\n",
|
||
" * Lower Hessenberg if $a_{ij}=0$ for $i < j+1$\n",
|
||
"\n",
|
||
" * Tridiagonal if $a_{ij}=0$ for $|i -j| > 1$\n",
|
||
"\n",
|
||
" * Lower banded with bandwidth $p$: $a_{ij}=0$ for $i > j+p$\n",
|
||
"\n",
|
||
" * Upper banded with bandwidth $p$: $a_{ij}=0$ for $i < j+p$\n",
|
||
"\n",
|
||
" * Banded, block upper triangular, block lower triangular....\n",
|
||
"\n",
|
||
"Some Equivalent Statements. For an $N\\times N$ matrix $\\mathbf{A}$ the following properties are all equivalent\n",
|
||
"\n",
|
||
" * If the inverse of $\\mathbf{A}$ exists, $\\mathbf{A}$ is nonsingular.\n",
|
||
"\n",
|
||
" * The equation $\\mathbf{Ax}=0$ implies $\\mathbf{x}=0$.\n",
|
||
"\n",
|
||
" * The rows of $\\mathbf{A}$ form a basis of $R^N$.\n",
|
||
"\n",
|
||
" * The columns of $\\mathbf{A}$ form a basis of $R^N$.\n",
|
||
"\n",
|
||
" * $\\mathbf{A}$ is a product of elementary matrices.\n",
|
||
"\n",
|
||
" * $0$ is not eigenvalue of $\\mathbf{A}$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0a85f2b2",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Numpy and arrays\n",
|
||
"[Numpy](http://www.numpy.org/) provides an easy way to handle arrays in Python. The standard way to import this library is as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "7043c92e",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[-1.25902112 -0.51174395 -0.29276615 1.6489862 -1.69115646 1.62620724\n",
|
||
" 0.65444431 -1.35346808 -0.20316225 1.12630042]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"x = np.random.normal(size=n)\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2b5abf29",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Here we have defined a vector $x$ with $n=10$ elements with its values given by the Normal distribution $N(0,1)$.\n",
|
||
"Another alternative is to declare a vector as follows"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "4193cb75",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1 2 3]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.array([1, 2, 3])\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b112c576",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Here we have defined a vector with three elements, with $x_0=1$, $x_1=2$ and $x_2=3$. Note that both Python and C++\n",
|
||
"start numbering array elements from $0$ and on. This means that a vector with $n$ elements has a sequence of entities $x_0, x_1, x_2, \\dots, x_{n-1}$. We could also let (recommended) Numpy to compute the logarithms of a specific array as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "18167a1f",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.94591015 2.07944154]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4, 7, 8]))\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ee482a57",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Here we have used Numpy's unary function $np.log$. This function is\n",
|
||
"highly tuned to compute array elements since the code is vectorized\n",
|
||
"and does not require looping. We normaly recommend that you use the\n",
|
||
"Numpy intrinsic functions instead of the corresponding **log** function\n",
|
||
"from Python's **math** module. The looping is done explicitely by the\n",
|
||
"**np.log** function. The alternative, and slower way to compute the\n",
|
||
"logarithms of a vector would be to write"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "d8305632",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1 1 2]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"from math import log\n",
|
||
"x = np.array([4, 7, 8])\n",
|
||
"for i in range(0, len(x)):\n",
|
||
" x[i] = log(x[i])\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a50fea8e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We note that our code is much longer already and we need to import the **log** function from the **math** module. \n",
|
||
"The attentive reader will also notice that the output is $[1, 1, 2]$. Python interprets automacally our numbers as integers (like the **automatic** keyword in C++). To change this we could define our array elements to be double precision numbers as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "e9263103",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.94591015 2.07944154]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4, 7, 8], dtype = np.float64))\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d68631ad",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or simply write them as double precision numbers (Python uses 64 bits as default for floating point type variables), that is"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "074cfbda",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.94591015 2.07944154]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4.0, 7.0, 8.0]))\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "dab5cce4",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"To check the number of bytes (remember that one byte contains eight bits for double precision variables), you can use simple use the **itemsize** functionality (the array $x$ is actually an object which inherits the functionalities defined in Numpy) as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "9d335488",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"8\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4.0, 7.0, 8.0]))\n",
|
||
"print(x.itemsize)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "64c9f3e5",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Having defined vectors, we are now ready to try out matrices. We can define a $3 \\times 3 $ real matrix $\\hat{A}$\n",
|
||
"as (recall that we user lowercase letters for vectors and uppercase letters for matrices)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "a3b32334",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[1.38629436 1.94591015 2.07944154]\n",
|
||
" [1.09861229 2.30258509 2.39789527]\n",
|
||
" [1.38629436 1.60943791 1.94591015]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"A = np.log(np.array([ [4.0, 7.0, 8.0], [3.0, 10.0, 11.0], [4.0, 5.0, 7.0] ]))\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4ff9ce6b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"If we use the **shape** function we would get $(3, 3)$ as output, that is verifying that our matrix is a $3\\times 3$ matrix. We can slice the matrix and print for example the first column (Python organized matrix elements in a row-major order, see below) as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "d19f85e2",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.09861229 1.38629436]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"A = np.log(np.array([ [4.0, 7.0, 8.0], [3.0, 10.0, 11.0], [4.0, 5.0, 7.0] ]))\n",
|
||
"# print the first column, row-major order and elements start with 0\n",
|
||
"print(A[:,0])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4e7e8796",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We can continue this was by printing out other columns or rows. The example here prints out the second column"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "4733c8b7",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.09861229 2.30258509 2.39789527]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"A = np.log(np.array([ [4.0, 7.0, 8.0], [3.0, 10.0, 11.0], [4.0, 5.0, 7.0] ]))\n",
|
||
"# print the first column, row-major order and elements start with 0\n",
|
||
"print(A[1,:])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "df9f205b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Numpy contains many other functionalities that allow us to slice, subdivide etc etc arrays. We strongly recommend that you look up the [Numpy website for more details](http://www.numpy.org/). Useful functions when defining a matrix are the **np.zeros** function which declares a matrix of a given dimension and sets all elements to zero"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"id": "19757d00",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"# define a matrix of dimension 10 x 10 and set all elements to zero\n",
|
||
"A = np.zeros( (n, n) )\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f1911274",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or initializing all elements to"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"id": "4f737773",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"# define a matrix of dimension 10 x 10 and set all elements to one\n",
|
||
"A = np.ones( (n, n) )\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cd241572",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or as unitarily distributed random numbers (see the material on random number generators in the statistics part)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "54645585",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[0.26185107 0.82454365 0.97434186 0.56556315 0.58187347 0.72509099\n",
|
||
" 0.19158446 0.3263505 0.0536097 0.90066122]\n",
|
||
" [0.34678929 0.66981186 0.44152248 0.20299677 0.1614891 0.68485505\n",
|
||
" 0.43333886 0.45741697 0.31311243 0.88001352]\n",
|
||
" [0.29661191 0.05268304 0.98153145 0.9007164 0.69481746 0.35319678\n",
|
||
" 0.88063413 0.06319374 0.06695337 0.75350216]\n",
|
||
" [0.15089627 0.58671946 0.13734823 0.72394787 0.38019139 0.422275\n",
|
||
" 0.35821426 0.49282737 0.19144544 0.84653115]\n",
|
||
" [0.38637915 0.8049181 0.49672291 0.98699753 0.8192798 0.05850532\n",
|
||
" 0.00152188 0.13131825 0.31229747 0.40183706]\n",
|
||
" [0.8705211 0.1472032 0.22567203 0.55202922 0.62683307 0.41566661\n",
|
||
" 0.23659936 0.85086629 0.85120833 0.79135075]\n",
|
||
" [0.31377492 0.38629844 0.33956555 0.64064128 0.42028578 0.58474054\n",
|
||
" 0.71760245 0.51732028 0.31211671 0.35894575]\n",
|
||
" [0.78286771 0.72594302 0.22821344 0.23962594 0.48739546 0.59190877\n",
|
||
" 0.8557822 0.44830642 0.90595152 0.9626883 ]\n",
|
||
" [0.34498451 0.90694878 0.15442554 0.43560678 0.89045167 0.21654926\n",
|
||
" 0.00355118 0.18695705 0.61123608 0.7386068 ]\n",
|
||
" [0.30813073 0.26549135 0.96812218 0.94321297 0.81592628 0.60980325\n",
|
||
" 0.4284066 0.8792323 0.92968793 0.82073684]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"# define a matrix of dimension 10 x 10 and set all elements to random numbers with x \\in [0, 1]\n",
|
||
"A = np.random.rand(n, n)\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3e993bf6",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"As we will see throughout these lectures, there are several extremely useful functionalities in Numpy.\n",
|
||
"As an example, consider the discussion of the covariance matrix. Suppose we have defined three vectors\n",
|
||
"$\\hat{x}, \\hat{y}, \\hat{z}$ with $n$ elements each. The covariance matrix is defined as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4fe66190",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\hat{\\Sigma} = \\begin{bmatrix} \\sigma_{xx} & \\sigma_{xy} & \\sigma_{xz} \\\\\n",
|
||
" \\sigma_{yx} & \\sigma_{yy} & \\sigma_{yz} \\\\\n",
|
||
" \\sigma_{zx} & \\sigma_{zy} & \\sigma_{zz} \n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cdd238d5",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"where for example"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "41663b63",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\sigma_{xy} =\\frac{1}{n} \\sum_{i=0}^{n-1}(x_i- \\overline{x})(y_i- \\overline{y}).\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e7bd73e9",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The Numpy function **np.cov** calculates the covariance elements using the factor $1/(n-1)$ instead of $1/n$ since it assumes we do not have the exact mean values. For a more in-depth discussion of the covariance and covariance matrix and its meaning, we refer you to the lectures on statistics. \n",
|
||
"The following simple function uses the **np.vstack** function which takes each vector of dimension $1\\times n$ and produces a $ 3\\times n$ matrix $\\hat{W}$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c4d0878",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\hat{W} = \\begin{bmatrix} x_0 & y_0 & z_0 \\\\\n",
|
||
" x_1 & y_1 & z_1 \\\\\n",
|
||
" x_2 & y_2 & z_2 \\\\\n",
|
||
" \\dots & \\dots & \\dots \\\\\n",
|
||
" x_{n-2} & y_{n-2} & z_{n-2} \\\\\n",
|
||
" x_{n-1} & y_{n-1} & z_{n-1}\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "32eb00e7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"which in turn is converted into into the $3 times 3$ covariance matrix\n",
|
||
"$\\hat{\\Sigma}$ via the Numpy function **np.cov()**. In our review of\n",
|
||
"statistical functions and quantities we will discuss more about the\n",
|
||
"meaning of the covariance matrix. Here we note that we can calculate\n",
|
||
"the mean value of each set of samples $\\hat{x}$ etc using the Numpy\n",
|
||
"function **np.mean(x)**. We can also extract the eigenvalues of the\n",
|
||
"covariance matrix through the **np.linalg.eig()** function."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 14,
|
||
"id": "6b91d50d",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"-0.2246674023625205\n",
|
||
"3.345687771875474\n",
|
||
"-0.6084611325305795\n",
|
||
"[[ 1.17709473 3.57810065 3.67258699]\n",
|
||
" [ 3.57810065 11.8548082 11.12152272]\n",
|
||
" [ 3.67258699 11.12152272 15.62249103]]\n",
|
||
"[26.06969872 0.07319349 2.51150176]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Importing various packages\n",
|
||
"import numpy as np\n",
|
||
"\n",
|
||
"n = 100\n",
|
||
"x = np.random.normal(size=n)\n",
|
||
"print(np.mean(x))\n",
|
||
"y = 4+3*x+np.random.normal(size=n)\n",
|
||
"print(np.mean(y))\n",
|
||
"z = x**3+np.random.normal(size=n)\n",
|
||
"print(np.mean(z))\n",
|
||
"W = np.vstack((x, y, z))\n",
|
||
"Sigma = np.cov(W)\n",
|
||
"print(Sigma)\n",
|
||
"Eigvals, Eigvecs = np.linalg.eig(Sigma)\n",
|
||
"print(Eigvals)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 15,
|
||
"id": "6a0aa964",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[1. 0. 0. 0.]\n",
|
||
" [0. 1. 0. 0.]\n",
|
||
" [0. 0. 1. 0.]\n",
|
||
" [0. 0. 0. 1.]]\n",
|
||
" (0, 0)\t1.0\n",
|
||
" (1, 1)\t1.0\n",
|
||
" (2, 2)\t1.0\n",
|
||
" (3, 3)\t1.0\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAACUoUlEQVR4nO29eXhU5d3//z4zmSXrZN8gGyGsQcUoECxuARS3LipaW9SnSGutRqA+rXT5uXy/LdWnNRiXqn1Q2mrFtojabykaIqDIokBAQZaB7CQh+zrJrOf3x5n7ZCb7MstZPq/rmovJzJnJPcyd+37fn5XjeZ4HQRAEQRCEgtAEewAEQRAEQRC+hgQOQRAEQRCKgwQOQRAEQRCKgwQOQRAEQRCKgwQOQRAEQRCKgwQOQRAEQRCKgwQOQRAEQRCKgwQOQRAEQRCKIyTYAwgGLpcLdXV1iIyMBMdxwR4OQRAEQRBjgOd5dHV1ITU1FRrNyDYaVQqcuro6pKWlBXsYBEEQBEFMgJqaGkydOnXEa1QpcCIjIwEI/0FRUVFBHg1BEARBEGOhs7MTaWlp4j4+EqoUOMwtFRUVRQKHIAiCIGTGWMJLKMiYIAiCIAjFQQKHIAiCIAjFQQKHIAiCIAjFQQKHIAiCIAjFQQKHIAiCIAjFQQKHIAiCIAjFQQKHIAiCIAjFQQKHIAiCIAjFQQKHIAiCIAjF4VeB88knn+DWW29FamoqOI7De++9N+pr9u7di7y8PBiNRkybNg2vvPLKoGu2bduGOXPmwGAwYM6cOdi+fbsfRk+onaKSsyguNQ/5XHGpGUUlZwM8IoIgCGKs+FXg9PT04NJLL8WLL744pusrKipw0003YcmSJSgrK8MvfvELFBYWYtu2beI1Bw4cwF133YVVq1bh+PHjWLVqFVauXIlDhw7562MQKkWr4fDcECKnuNSM50rOQquhTvTEYEgYE+OF5ox/8GsvqhUrVmDFihVjvv6VV15Beno6Nm3aBACYPXs2Dh8+jN///ve4/fbbAQCbNm3CsmXLsGHDBgDAhg0bsHfvXmzatAlvv/22zz8DoV4KC3IAAM+5F5fCghxR3KxfNkN8niA8YcIYEOZMkYcYZnOHUVxqhtPFY53HY4T6oDnjHyTVbPPAgQNYvny512M33HADNm/eDLvdDp1OhwMHDmDdunWDrmGiaCisViusVqv4c2dnp0/HTSgXT5Hz/C4znDyPxdlxQ4obWngIwHvOnG7oQkqUEZs/qwAAL2HsKZYJdeM5Z7r7HNCFcHi+9DwAmjOTQVJBxg0NDUhKSvJ6LCkpCQ6HA83NzSNe09DQMOz7bty4ESaTSbylpaX5fvCEYlmcHQcAcPI8AGD/+RY8/a+TXteQ24rw5MFrspEWE4odX9WL4gYAOnrtAECWQGIQhQU5uHleCl77tBwv7T4vPr7/fDM+r2ilOTMBJGXBAQa3QOfdm4rn40NdM1Lr9A0bNmD9+vXiz52dnSRyiEEws7Dn4nGkqg33/GlwfNfrn1XiQHkLXlt1BbaXXaCFhxDheR6/3P4Vatp6Bz23eV8FXv+sAjwPmi+EF+eburHnTOOgxw+Wt2LlqwcA0JwZL5ISOMnJyYMsMY2NjQgJCUFcXNyI1wy06nhiMBhgMBh8P2BCUQz0gx+tbsN3XzsIm9MFAHjk+ulo6rJi6xc1AIBT9V1Y8uxuALTwqJWhRPHLe87jH0dqxZ91Wg52J4+0mFDUtPWC5zHoNYS66bU58dCbR9FjcwLonzPXzEjAJ+YmuM/5+ME3soI4SvkhKRdVfn4+SkpKvB776KOPcMUVV0Cn0414zeLFiwM2TkKZFBbkYP2yGXiu5Cw2vPsV7n7VW9z8dPlM/O72Swb5v2mzUi8DM+3eP3YB//PhGfH5xdlxMP/mJqxfNsPLouN08Xh25+mAj5cIPgMzpniex6/eO4EzF7sAAHnp0eKc2Xu2X9wAwP2vfx7o4coavwqc7u5uHDt2DMeOHQMgpIEfO3YM1dXVAATX0b333ite/+CDD6Kqqgrr16/HqVOn8Prrr2Pz5s147LHHxGseffRRfPTRR3jmmWdw+vRpPPPMM9i1axfWrl3rz49CqITCghw88I0svP159SBx43kNi8sBhM3qDx+dGfRehPLxFMU/3/Yl/vsfX4rPLc6Ow9/WLBr0mpgw4bD28p7zw6YGE8ploCj+++EabDvab/G7ZmbioNd8a34qAOBwVRue/OBEYAaqAPzqojp8+DCuu+468WcWB3Pfffdhy5YtqK+vF8UOAGRlZWHHjh1Yt24dXnrpJaSmpqK4uFhMEQeAxYsXY+vWrfjVr36FX//618jOzsY777yDhQsX+vOjECrCM1BYp+W8xA0gBIjuP9+CRwty8G5ZLWpae/HCx+eg02rIkqNCCgtyYHO48OLuc+Jj+dNiRXEzMPPFfLEL//qyfpBLlFAHnhlTjV1W/ONwjfgcc3UPnDNOF4+s+DBUNFuwZX8VYsL0eHQppY6Phl8FzrXXXisGCQ/Fli1bBj12zTXX4OjRoyO+7x133IE77rhjssMjiEFYHU785UAVACBEI/jBi0vNQ6ZpFhbkYEpMKH72zy8RqtPSZqVi0mJDxfs6LYe3f5gv/ux08V4xWjzP40h1G+ra+zAnJQpOl/caSZuV8iksyAHP8yja1W/BW7c0R5wjA+cMAHT3ObC5WcjIO9PQJT5OqePDI6kYHIIINuu2HkOv3YkIQwhO/58bRfcDMycPXHi+M38KMuLC0Gt34hvT4wdtVoQ6eN69UWm5flHMWDdgo+I4DtfMENwQX9d34oa5yeJzVG5APVw3q98VpdNyXhaZgXMGAH596xwsmhYLAPjE3Iweq4NSx0dBUllUBBFMikvN2HFCyNBbs2QaQjxcTsw6M/BU/cLH5zA7OQpVLRacqOvAH79/udf70Ulc+Tz5wUnUdfSB44ADG67H1i9qRrXmbfzOPBypasXZi9348ZtHsPdn19FmpTKeeF+opaXhMMhSPBxb/msBFvx2Fzp7HZj35IdwUbmBESELDkG4aerqAyDE4Nx1ZX+dJBZIOpR1RqvhsPNkA2LCdGi32PHn/ZUA6CSuFopLzdji/s4LZiUiMcroFXg8UhDx/957JTQcUNVqwfRf7CBxoyKKSs6irKYdALD5vivHNF8AwKjT4n/uuBQA4OIFyw/Nl+EhCw5BuAnRCnq/YFYikk1Gr+eGW0QGWnhe+6QcvXYnXtp9njYrFWB3uhCm18Jic+KOPG9RDGBEl2V6XBjuW5yJNz6rhMPFQ09B6qqguNSM591CJj5CjyU58aK7aixxfKfr+1sNjdXyo1ZI4BAEhEJb29zF2e5ZmD6u1xYW5MDF89i0y4zOPgeJGxVxWVo0LDYn4sL1uH6Wd3rvWL5/zyQMm9NFm5UKcLp4zEiKwNmL3bj10lTxYDUWUVxcakbRLjMumWrCl7UdyJ0SRckNI0AuKkKVDCy29f++rENnnwNTY0JxvKYdRe5FY6ysXTpDdEdxoMVGLfzdneL77flToA8Z33IquLeqEO2ui3PzvJQxuSkIebN6SRaqWiwAgO/Mn+r1XGFBzrAxe54xWj+/cRYAoK69D2sLcmjeDAMJHEKVDCy29bfPhXpMmXFhKNplHnfsDAsoBgAeoCq1KqC524rSU0LvoDuvGF9vO8/NaqX7tSFabsyxGIR82flVA6wOF6YnRiB3StSYX+eZwbkgKxbRYTq09tiwcFrcsDGCaodcVIQq8S621Yey6nZoOGDfuZZxu5c8N6t/Ha+DubEbL+85D6NOS5YcBTGw79R7ZRfgcPG4NC0aH55swI6v6secMee5WR2ubMVrn5Rj9+lG/P7OS8XnCWXybpngCv/2/CkjNokeiOfc0mk1WDY7Cf84UoudJ+rx1DdzfT5OJUACh1AtAwOEJ5JyOTC1t8fmgLmxG7OSI8k3rjA8Kw8/cv10vONuuhofoR93oTXPzWp+egziwvVo6bHh84pWmi8Kw1MYX2jvxcHyVgDAt+ZPmVQpiRXzkgWBc7IBT9w6FxrK2BwEuagIVfPwddPF+yETaJo5sPBfwSyhq31Dp+Abp5O4cvBM//7F9hMwN3ZDq+FQeqpxUkHlWg2HgtlCgHLJ1xd9OWRCAni6w98/dgEAsGhaLLYdqZ1UKYmrpscjwhCCi51WMeWc8IYsOISq+f8+OCned7jGn3I58OR1eXo0TKFCTZyrcuJxZWasz8ZKBJ+BVr+hSupPhGVzkvH3w7X46GQDnrh1zrhcF4S08ZwzseF6AECkMWTSdY8MIVpcPysRHxyvw4cnG5CXEeOzMSsFsuAQqqW41Iw3Dwp9p66dmeCTAM8QrQbXzkwAADEAlVAWnhvSRKx+Q/GN6fEw6jSo6+jDybrO0V9AyIrCghx8b2E6WntsAICSrydn9WOsyBXafPznRP2IfR/VCgkcQpWw2JmsuDAAwFXZ8WOuQDsarB7K7tMkcJTIb/99SrzPrH6TJVSvxZIcQRiTm0qZMOsNAJ8UdSwqOYuv6zth1GlQ09rrJYyLS83jLnWhREjgEKrE6eLxaEEOGrusAID87DgAI7dlGCvXzEiAVsPhzMUu1LRafDJeQhoUl5rx2qflAIB5U0w+TetePkeI3yKBo0zePSrE32g1nFjUcTJoNRxe+PgcpkYLnew/PCn00aM2Mf1QDA6hStYtm4HDla14vtSMmDAd5qT016OY7MkqOkyPvIwYfF7Rit1nGnFvfuYkR0tIAbZx5KZG4URdJ/Kz4wbF5Ex07hSVnIXN4YKGEzqM17ZZMDUmTPy91LRV3jz30RlcaO8FAJSsuxr/78v6Sc+ZgXPvPycaoNNqqKeZB2TBIVTLZ+daAAjWG1+nWBa43VQUh6McWEBxZ58DgJAJA/jG6qfVcPjj3vNIMQmn8V1uKw6dxuVPcakZxR+fAwAkRBqQFR/uM3d4YUEOfnJdNgDgXGM3iZsBkAWHUC2fnW8GACzOjvfp+xaVnEVHrxBMeOB8C3qsDoQbhD81Oo3Ll3XLZuBCe68oODwz5Ca7oQw8jZecuojOPgdtWArA6eKRPy0OB8pbsDArVsyQG0vvqbHw3zfMwst7zoPn4VWIkiALDqFSem1OlFW3ARDqSfgSrYbDlv1VMIXqYHO68Nk5QUjRaVz+HDwvWP1yp5gQadT59L0LC3Lwg6syAQjWRRI3ymDdshnQuHfahVneZSNG6j01VopLzWAJVE4fBb0rBRI4hCr5orIVdiePVJMRme5MKl/BzM8dvXYAwMenGwdVPCbkyYFyQeAw95Sv+f9unSve91UKOhFcbA4XjlQJh6mF0+J8+t5sXbnrSqGfWZheS73MPCCBQ6gS0T01Pd4vRdUKC3Jw++VTAABbv6ghcaMQDrgtOPk+3qgYnhuTr1LQieDy1YUO9NldiA3XIycxwmfv63loeuq2udBpOVhsTqz+RhaJHDckcAhVst8dYHzVdP9sVADw2+/ME+/rtHQalzs1rRZcaO9FyID4G1/BNqxl7rYNU2NCaaNSAIcqhLXmyswYnx6mPKtoG3Va5E4xAQByp0RRd3E3FGRMqI52iw0n6joA+D7A2JNX95aL9+3O8beBIKQFc09dMtUkBo37Cs/T+I25ySg51YiWbhvWFuRQ01aZc8jdXHNhlm8PUwNjd/LSY1BW3Y4jVW34v9+aN8yr1AVZcAjVcbC8BTwPTE+MQFKU0S+/g21Yl6dHAwAuSzPRaVzmsADjRX5wT3mexqcnRCDSEIJeuxNL5yTRaVzGOJye8Tf+7Ut3RabQi+pwZZtff4+cIAsOoQqK3NlLhQU5Yv2bxe7qxb5O3fY8jWfGh+NodRl4HmLdC4BO43KD53kcLO+vm+RrPOeeRsPhsvRofGpuRllNO80VGfN1fSe6rQ5EGkMwKzlq9BdMgsvTBYFz5mIXuvrsPs/ykyNkwSFUgVbDiRYUz/o3/kjd9jyNz0+LBiAsdD+8ehqdxmVKdasFdR190Gm5gHRtnu/erMqq6DQuZz6vENxTCzJj/V4eIjHKiLTYUPA8cKym3a+/Sy6QBYdQBQMLqWk44Mvadry857zPs5s8T+NTY0IRF65HS48NJ+s66TQuIzytfix76tKp0QjTh/i9YCNzbR6tJoEjZw66428WZPnXPcXIS49BTWsvDle2ic1b1QxZcAjVUFiQgxW5yQAAnodfxM1AOI7DfPdmRacqeeFp9fN0TwWiYOP8NMGCU9liQWuPzW+/h/AfLhePLyrdAcZ+KiswkDx3dh8JYwGy4BCqggUV8wD0Wk1ALCqXpUVj16lGEjgyw9PqF27QAgDqO/rwzyO1fhfGpjAdshPCcb6pB2XVbSiYneS330X4Dk+r35mLXejotSNMr0VualRA2rTkMddmdTucLl71VdPJgkOoipKvGwAIp3Ob0xWQrKbL3KfxYzV0qpIbhQU5+K+rMtFjdQJAQMQNgwWN0mlcPnha/Q65rX55GTF4ec/5gLRpmZkciQhDCLqtDpxp6PLr75IDZMEhVMOmXWdxob0PALDz0SX4z4mGgGQ1XZJmAscBNa29aO62Ij7C4LffRfiey9yB4kDgrH4AcHlGDP5xpBZHq9oD8vuIyeNp9WNVi108H7BK5lqN4BL/1NyMI9VtmJPq38wtqUMWHEIVFJeasWmXYK0J1WkxLSFC7Bnl7/o0UUYdshOExe5Ydbvffg/hH/56oAqAEJgeKKsfADF263htO2XeyYjCghysW5oDc2M3AKFxaiDbtIiWP8rAC4zAefnll5GVlQWj0Yi8vDx8+umnw157//33g+O4Qbe5c/ub0G3ZsmXIa/r6+gLxcQgZ4nTxYoDxnNQo0VTMRI6/NxCWLk5xOPKiuNSMw+6NYuN35gVEEDNyEgV3g8XmJHeDzLg9b6p4P9BtWlgZg8NVrQH7nVLF7wLnnXfewdq1a/HLX/4SZWVlWLJkCVasWIHq6uohr3/++edRX18v3mpqahAbG4s777zT67qoqCiv6+rr62E0+qcqLSF/1i2bgRRTKABgnrtnC6OwIMevgX8AcBllUskOli1lCBGWybmppoBZ/QDB3cDcYxSHIy+e3XkGAMChv01LICgqOYtDFS2iS7yxs//QX1xqRpHbJa8W/C5wnnvuOaxevRoPPPAAZs+ejU2bNiEtLQ1//OMfh7zeZDIhOTlZvB0+fBhtbW34r//6L6/rOI7zui45OdnfH4WQOScuCP2ncgcInEDANqrjNe1wkbtBFjhdPFZ/IwtWhws6LYcZSZEAAmf1A/rdVGXk2pQNxaVmfHC8DoBgyQmk1U+r4fDS7vOIC9cD6BfGgShtIEX8GmRss9lw5MgRPP74416PL1++HPv37x/Te2zevBlLly5FRkaG1+Pd3d3IyMiA0+nEZZddhv/zf/4P5s+fP+R7WK1WWK1W8efOzs5xfhJC7rhcPE66G2wOtOAEgplJkQjVadFldeB8Uzdy3JslIV3WLZuBnSfqsXlfBWYkRUIf0n8eDHQ8RRlZcGQBExLT4sNR3tyDualR+K+rsgAgIAkNAwuaHq5sw9mL3QELcpYafrXgNDc3w+l0IinJu4ZDUlISGhoaRn19fX09/vOf/+CBBx7wenzWrFnYsmULPvjgA7z99tswGo246qqrYDYPrZA3btwIk8kk3tLS0ib+oQhZUt7cgx6bE0adBtkJ4QH//SFajSisyshNJRtOXBAOQ7mpgRfFRSVnxUJx5c09aPMo+KdGd4McYG1a+uxCWYE5KUIWUyCtfp4FTTfvq1CtuAECFGTMcd5mMZ7nBz02FFu2bEF0dDS+9a1veT2+aNEifP/738ell16KJUuW4O9//ztmzJiBF154Ycj32bBhAzo6OsRbTU3NhD8LIU+Y9WZOShRCtMFJHqSKxvLjRB1zawY+3Var4fDynvOICROaJrJ5o1Z3gxxYt2wGVi3KQF2HEPvimaYdiFg/xuMrZgEQCpoGOshZSvjVRRUfHw+tVjvIWtPY2DjIqjMQnufx+uuvY9WqVdDr9SNeq9FocOWVVw5rwTEYDDAYqPaImvmqNnjxN4BwGq9r7wUwOJ4iEBVOiYlxsk6w4MwNwrwZ6G44Wt2Gry50qPpELge+rhfmTEZcWNA6er9XdkG8z4Kc1Thf/HqU1ev1yMvLQ0lJidfjJSUlWLx48Yiv3bt3L86dO4fVq1eP+nt4nsexY8eQkpIyqfESyuWrIAYYA8Jp/F9f1gMAzjR0wmJzAKDTuJRp7OxDU5cVGg6YnRycgmmFBTkomJ0IAHjx43MkbmQAsxbPDVKRveJSM4p2mZFqErKKV+QmByzIWWr4vZLx+vXrsWrVKlxxxRXIz8/Ha6+9hurqajz44IMABPfRhQsX8Je//MXrdZs3b8bChQuRm5s76D2feuopLFq0CDk5Oejs7ERxcTGOHTuGl156yd8fh5AhQoCxcKoKRoAx4H0ad/GCRelQRSttWBKGuaeyEyIQqtcGbRyPFuSg9FRjQPunEROHrTUs/iaQsAPT+mUz0NRlxV8PViE9LkzM5AICFyAvBfwucO666y60tLTg6aefRn19PXJzc7Fjxw4xK6q+vn5QTZyOjg5s27YNzz///JDv2d7ejh/+8IdoaGiAyWTC/Pnz8cknn2DBggX+/jiEDKls6UG31QFDiEYsnx4MCgty8MGxOpxr6sY9fzoEJ8+TuJEwYoBxkEQxo+Tri+J9VkmZ5ox0+Zq5NYMQmM6CnAsLcvDWIaEC96n6LvzlBwvE59VEQHpRPfTQQ3jooYeGfG7Lli2DHjOZTLBYLMO+X1FREYqKinw1PELhMPfU7CAGGDO+kzcFz+48AyfP02lc4gTb1QAIJ/IXPj6HmDAd2ix23H75FFWexOVCr82J801Ci4ZgzBvPOL7ZbgvSaXdMkBrnC/WiIhQPK/AXLPeUJ2cvCiX3OQS2rxExfpgFJxgnccDb3ZCfHQcAmJUcFdDCccT4ON3QCRcPxEcYkBgV3Mr6M5MiwXFAY5cVLd3W0V+gQEjgEIrnK4kInOJSM94rEyqcajUcHi3IoY1KorT12HDBnfUWrI7Mnu6GWe4g59MNXQGtqUKMDzH+RgJdvMMNIciIDQMgzBs1EhAXFUEEC5eLx0l2Eg9CLRMGO42vW5qD/91Xga4+B1bMS4ZWw5HLQYKwjSojLgym0OCk+nq6G2YmC5WvTzeo190gB1iKeDDdmp7MSo5CZYsFp+o7cdX0+GAPJ+CQBYdQNNWtFnRZHdCHaMReQsGAncYfXToDM93jOEOnccnC4m+CUcF4KFiaurmxGw6nK8ijIYZDrJskFYGTIqw1p+rJgkMQiqDIXVemsCCnP8A4ORI6rSZoRfU8f9+M5EgcrmrD6YYufBN0GpciJ+qCb/XzZGpMKML1WvTYnKhs6cH0ROplJjUcTpcY0BusuK2BsEDjU/Xq7L9IFhxCcTC3T3Gp2auDuFSK6s1K7rfgENLk5AWWQSWNjUqj4TAjWd2ncalT3twDq8OFcL1WjH0JNszyd66xG3YVWv7IgkMoDs+iemkxoQCEqrRvHaqWRN0ZTxcVIQ08rX5dfXaUN/cAEFwNUmmlMSs5CmXV7Tjd0IlbL00N6liIwbD6N7NToqCRSGXyqTGhiDCEoNvqQHlTjxjLpRbIgkMoksKCHKxbmoOaNiETpuRUoyTEDQAxI+ZCey+6+uxBHg0BeFv9mIUkxWTE3w5VS8LqBwCzU0gYSxkp1E0aiEbDDQpQVxMkcAjFcveCdPG+lDrqmsJ0SHbXyGB1cYjgwoK9nys5i5f3nAMAhOq0kmqlwSx/5KKSJieDWMF4JGarONCYBA6hWJ7deUa8zzrqSoX+U5X6Fh2pwkTOnjNNAISYCqmIG8Db8tdJlr+gU+RRw4rnea8aOMWlZhS5yz8EGzZv1BhoTAKHUCTFpWZsO1oLAFg+J0ly1V8p0FiaFBbkgDmjQjTSsfoBguUvxd0h+izNm6Dj6das6+hDR68dIRoOH33dIBm3JuDRskGFLioKMiYUB8uWmjfFhK8udGBGUqRX4DEQ/NRsVpOHLDjSYtOus2AViRwuXnKNLWclR6K+ow+nGrpwRWZssIejajzXlHONQv+p6DA9ikvPScryx6zFFzutaO2xITZcH+QRBQ6y4BCKgxXVM4QI0zsnSeggLqWiejM9LDg8H/zxEIIw3rRLsPDpQzRYu1R6rTRmDWigSAQXtqZ8cFxowdLcbZWUuAGACEMIMuLcLRtUNm9I4BCKY92yGXjk+ukwu09VOR5F0QoLcoKe7gsA0xMjoNVw6Oi1o7FLnY3wpASz+n3rMiH9OjshAmuXziDXJjEqhQU5YN4orcTcmgw2b74mgUMQ8qepy4qOXjs0HDAtITzYwxmEUadFZpy6G+FJCWb1YxaS6YnSs/oB8Gq6SZY/aVBcagabHk6XtJIZGP1xOOpaaygGh1AkzHqTERcOo04b5NEMzazkKJxv6sGZhk5cMyMh2MNRNcyq99O/HwcA5LgFDhD8eC1PpiWEQ6fl0G11oLatF2kSqZirVpjlT8MBLh544BtZkonz80StmVRkwSEUidldX2a6x0YlNShVXHqcaxS+ixyJzhudVoPsBGFs5KYKLkzc3L84Ey5eqJv0i5tmS8qtyVLZ57gtOOaL/c1apZTK7i9I4BCK5KzbgjMjSZobFeAdaEwEH57nxWyYHAnPGzWn/UoJ5tbMz44DAGQnhkPjjsGRiluTpbJvL6tFuF4Lm9OF8uYeyfTl8zfkoiIUybmLgwOMpQarTGtuFE5VIVo6bwST+o4+9NicCNFwyIiTXtwW65fFAkZPeQhjqfTLUhPs//ql3ULl64HJDFLAM5U9xWREj82JTSVnseNEg+SyvfwBraiE4uB5HmeZq0HCJ/H02DCE6rSwOVyoarUEeziqh8VtZcWHQydBsclO48ylySx/ajmNSxVm9ZOqO5xZlOo7+gBANeIGIIFDKJDmbhvaLUIGFYtXkCIaDSe60MhNFXxY3JZURTHbqLaXXQAAlDd147mPzkiqX5YaYQJHymtNYUGOKIA5TjoWJn9DAodQHGyjSo8Nk2wGFQv+GyrQWA3Bf1JEPIlLfKNat1TYnFw8UPyxtKrmqg2Xi8f5JunHbTEXJgDwPCQRAB0ISOAQisMsmoylG3/D3A0XO4Uif2fcAaPkbgge4rxJku68AYBHl84Q+2VJtbCcWqjv7IPF5oROyyFDoin7bE15YEkWAEDDQTJZXv6GBA6hOM5K3NUA9Lsb9p4VOlefaegSFyI6kQcenuf7XVQSjaVgFJeaxX5ZUi0spxbYnMmMC5dkkoDnmvKLFbMRqtPCxQP/tThTFSJHet+IDCkaYaKQuyHwmGWQIg4IIufBa6YBACpbLCRugkhTtxWdfQ5oOCHIWKqwDYsVhsxJjFDFRiVVpB5gzFLZCwtyoNFwYlX3xdPjJZPK7k9I4PgA5m4YuMiQuyHweJ/Epe1qAIDHV8wW74eQuyFosLICUq587Xka/6+rMgEIAaNSKiynNsT4G4kKnHUDDkwsEPp8U7dk+vL5E6qD4wM8aw3wPI/vL8rAW4eq6UQeBFp6bGiz2MFJPIOK4bkpOdzuBpovgccs8ZM44H0ar3GXFahstuAn100XnycCi5hBJeF544kocNzjVjokcHxEYUEOGjr7ULTLjKJdwqZF4ibwmN0n8bSYMITqpXkSZ7AT+dzUKJys68SiabGS7GOjBsyN0m/t4XnaTo0OhSFEA6vDhdo2C82XIMDzvCyEsSdsnMzypHTIReVD1i7tX2R0WnI3BAO2UUk9/sbT3XDbpakAgIRII7kbgoT5orRdDQPRajgxVqi8qSfIo1EnLT1CvS25WIsBoZ0EAJxv6lFFN3oSOD7k7UPV4n27k7IbggHbqKScIg54uxumuRfHcrdfXA3Bf1JD7EEl8XnjiWc8BRF42JxJi5Fuva2BZMaFg+OAjl47WnpswR6O3yEXlY8oLhVcU8lRBjR0WnHrJankbggCLEVc6hYcT3cDy2yoaBZOVTRfAktrj01c7NkJVw6weUMCJzjIzT0FAEadFmkxYahuteB8YzfiIwzBHpJfCYgF5+WXX0ZWVhaMRiPy8vLw6aefDnvtnj17wHHcoNvp06e9rtu2bRvmzJkDg8GAOXPmYPv27f7+GMPi6W5YNE3oLDs7NZLcDQHCM01/4ElcDmn66bFhCNFwsNicaOjsC/ZwVAebM1NjQhGml8+Zr9+CQy6qYHBehgIHALIT+t1USsfvAuedd97B2rVr8ctf/hJlZWVYsmQJVqxYgerq6hFfd+bMGdTX14u3nJz+U+2BAwdw1113YdWqVTh+/DhWrVqFlStX4tChQ/7+OEMytLuhh9wNAYKl6f/uP6fQ0mMDxwmLjlzS9HVaDdLdVVApniLwsLgtucTfMJgFp5wsOEFBDq09hkJNrk2/H1eee+45rF69Gg888AAAYNOmTfjwww/xxz/+ERs3bhz2dYmJiYiOjh7yuU2bNmHZsmXYsGEDAGDDhg3Yu3cvNm3ahLffftvnn2E0hnM3AOSeCgSeafqAcBL/06flskrTn5YQjvLmHpQ3deOq6fHBHo6qEAOMJd6iYSDsMNXcbUOHxQ5TmC7II1IXosCRuDt8INkqyqTyqwXHZrPhyJEjWL58udfjy5cvx/79+0d87fz585GSkoKCggLs3r3b67kDBw4Mes8bbrhh2Pe0Wq3o7Oz0uvmL/swG5U8eKVFYkIPrZwnVXWvbemUlboD+zUoNZmMpMJRbk53E5eDWBIAIQwiSooQYivPNtN4Eks4+u+hOlp+LigSOT2hubobT6URSUpLX40lJSWhoaBjyNSkpKXjttdewbds2vPvuu5g5cyYKCgrwySefiNc0NDSM6z03btwIk8kk3tLS0ib5yYaHCZw2ix1tKohSlxLpscL/Pc8Deq1GNuIGAKYxYdxMAicQeFYfF2vgJMnHrcnI9nCJE4GDxd8kRhoQZZSX5YzF4NS29aLP7gzyaPxLQCLqOM57seB5ftBjjJkzZ2LmzJniz/n5+aipqcHvf/97XH311RN6zw0bNmD9+vXiz52dnX4TOWH6EKSajKjr6EN5cw/ywvV++T3EYPacaQQgbF42p0tWVYE9U8UJ/zPQrQkApacu4qXd52Vm+QvH/vMtqjiNSwmp96AaidhwPaLDdGi32FHR3IPZKVHBHpLf8KsFJz4+HlqtdpBlpbGxcZAFZiQWLVoEs7k/Eyk5OXlc72kwGBAVFeV18ydZFPwXcIpLzahsEcrX/+2BhbLLYGOxWxfalX+qkgqFBTm4e0H/QUdu4gbwtODQWuNvvNyaA3pQycWtCQjGAbW4qfwqcPR6PfLy8lBSUuL1eElJCRYvXjzm9ykrK0NKSor4c35+/qD3/Oijj8b1nv5kWrx70SF3Q0BgbgVmv8uKDxcz2OQicuLC9YgyhoDn+wPUCf+Tlx4j3pebWxOg2K1A4unWPHex34IjN7cm4JEq3qjseeN3F9X69euxatUqXHHFFcjPz8drr72G6upqPPjggwAE99GFCxfwl7/8BYCQIZWZmYm5c+fCZrPhzTffxLZt27Bt2zbxPR999FFcffXVeOaZZ/DNb34T77//Pnbt2oV9+/b5++OMCTGTihadgOB08bh/cQa27K9CuF6LhEgh8JJtVnJI0+c4DtMSInCsph3lTco2G0uJfxyuAQBoOMjOrQn0b1RVLT1wOF0I0VJxen/h6dY0hQpxN8drO/DPI7Wytfwp3YLjd4Fz1113oaWlBU8//TTq6+uRm5uLHTt2ICMjAwBQX1/vVRPHZrPhsccew4ULFxAaGoq5c+fi3//+N2666SbxmsWLF2Pr1q341a9+hV//+tfIzs7GO++8g4ULF/r744wJMZOKMhsCwrplM1Dy9UVs2V+FrIRwr1gsOS060xLC3QKH5k0gKC414/PKNgDAL26aDYvNKbvq46mmUBh1GvTZXahp6xXXHsI/FBbkwOF0ofjjcwAgS3EDkMDxKQ899BAeeuihIZ/bsmWL188/+9nP8LOf/WzU97zjjjtwxx13+GJ4PodNnsoWC5wuXlamS7lS4RaTWfHyC/pjiPEU5KLyO8ytEB9hQHO3FdMSwnH9LCGGT04iR6PhkBUfgVP1nShv6iaBEwBuuiRFFDhydGsC/bVwypt64HLx0Ch0jyJ7ph9IjQ6FPkQDm8OFuvbeYA9HFbC4FTkv8NkUnB4wnC4e65bmoMfqACA0IQQgy+rj2dSTKqC85BY3HPrdmnIjLSYUOi2HXrsT9QpuDyOfxisyQqvhkBkXhrMXu3G+qRtp7jL8hP9gdUCy4uX7f+3Z5mOksgfE5Fm3bAYaOvpQtMsMrYbz+huV24l8GtXCCRjFpWb868t6AMBtl6UiOyFCVhY/RohWg8y4cJgbu3G+sRtTokODPSS/QBYcP8EyqSgjJjBUtjCBI18XVUZcGDQc0GV1oKnbGuzhKB4WIyecZuW7FJIFJzAwt2ZuqpAAkBknv2xNT9QQh0MWHD/R3wiPBI6/6bE6cLFTEARZcfJ1URlCtJgaE4bqVgvKm3qQGGkM9pAUTWWzUDcpU8ZuTYC6igcK1lR537lmAP3ucDllawJCPR+thkN2Yjhw0lvgFJeaBfetR39FOSPfY4vEoUyqwMGsZHHhetk3HCRhHDj6rX7yFjhs/K09NmoP40fWubOlKt3rjacwLizIkY0oYPV8WA0c9q8c6/mMBgkcP8H84lQLx/9UDLHgyBWxSKSCzcZSoT9uS97zJtwQghSTYO2jA5V/6bE60Nglb2sxc6vtPCl0Azjf1C2KGzmmvI8Euaj8BPOL13X0wWJzIExP/9X+olIBGVQM0YJDsVt+RwkWHNHdkBCB+o4+nG/qQV5GLADluRukAJszMWE6WVuLCwtyYHU48dLu82jssipS3ABkwfEb0WF6xLj/ACjQ2L8oIUWcMY0CRgOC08Wj2t27LFOmJ3Gg393QbbUD6J83SnQ3SAEWt6WEtea/b5gl3g/RcIoTNwAJHL8iuqlI4PgVZu2YpoBFhwWM1rRaYHVQ001/UdfeC5vTBb1Wg1QZp8gyd8Oxmg4AgttNqe4GKcAKiirBHe6Z9eVw8bLLAhsLJHD8CNtwKWDUvyglBqeo5Cy2fl6NcL0WLh6ihQGQV7diOcDmTEZcmOytHIUFObj98ikAgJKvL5K48SMVzIIjY6sf0G/hm+NOeV+cHSfLVPfRIIHjR7KoMq3faeuxoaNXMM/L2dUACO6Gol1mhBuEeC2W9kvuBt+jFFHM8IyzkWv7ADnAYnDkPG88LXw3z0sBAKSYQmVbz2ckSOD4ESr253+YeyrVZESoXhvk0UwO5m5gWRrlzcrNbgg2SorbAoSmjwy5tg+QA0pIaGD1fAoLcpARJ1TwrmzpkWWbktGg1B4/wDIbVuQmA/AuvU+ZDb5F3KgS5LvgeFJYkIMD51twoLwFv//wDFw8SNz4ASVkUDGKS83YtMuMmDAd2ix23H75VFm2D5A6nX12tLjrDMnZguO59zCrd5X770Fp84UsOH6AZTb868s6r9L75GrwPWLQn8zdU57cszAdAODiyd3gL0QXlcznjaeFLy8jBgBwWXq0It0NwYZZb+IjDIgwKMM2wCw4zd02dPXZgzwa30MCxw8wU19x6TlEGoVU8d9/eIZcDX5ASWmbjIPlLeJ9cjf4HpvDhdq2XgD9aflyxdvd4D6NNyvT3RBs+t2a8m3oO5BIow7xEXoAQJVHUoNSUIYMlSBMxDBT8d8P15K48QNiirjMNypGcakZbx2qFn9++Lrp5G7wMTVtFjhdPML0WiRGGoI9nEnh7W5g8RTCRkXzxbeIvctkbvUbSGZcOJq7bahs6UHuFFOwh+NTyILjRwoLcsC8URqOFhxf43LxHkF/8u0izvB0N7AikTfNSyF3g4+pFFPEw8FxynEXZwyIpyB8ixIyqIaCzZtKBSbDkMDxI8WlZjALsYsHbVA+5mJXH3rtTmg1HKbGyLdYG2NId4NCsxuCSYWCCkN6IgaMtlrgornic5SWeccYaPlTEuSi8hPsNP7t+VOwvewC4iP05GrwMayRaXpsGHRa+Wv1ge6GYzXt5G7wA/01cJQTSwEAqdFGhGg42BwuNHT2ybpCsxRRSmD6QJhFSomWP/nvChJkqNolPVYn1i3NIVeDD6lQUKrvQMjd4D/6U8Tl79b0JESrQXpsf10Twnd4FRRVmDBmgo1VaVYSZMHxA56uBpvDBa2GQ6/die8uSAfHceRqmASsxlBhQY5owWECR0k1htgiShuV7+mfN8raqAAh7be8uQeVzRYszg72aJQDO0wlRRkQplfWtpkRz1LFrei2OhSTAg+QwPELXmXTQzSYEh2K6lYLKlss5GqYJKzGEOBdbt/TaqYE+i04yjtVBZM+uxN1HX0AlOdqANi8aSLLn49RQgXj4Ygy6hAXrkdLjw2VzcrKpCKBEwAy4sJQ3WpBVUsPFmTFBns4ssYz/Z5lGh2vacc/jygrDZ9tvvUdfeizO2HUybsNhVRggjHKGILYcH2QR+N7MuPI8ucPlCxwAGGPaumxoarFoiiBQzE4AYBVi6TTuG8oLMjB2oIctFkEn7jSxA0AxITpEGkUzh/VrTRvJkORR9wbq3ydFR8utk5RUpf2jHiy/PmDihZl1sBhsEBjpQljEjgBgP1RKG3yBJNvXz5FvK/EdgYcx/XPGwXWpwgkzK1ZXGoWAyk93ZpKap3iudbwPMX6+YpKhXWfH4hS1xpyUQUAltlApyrf8bz7RM6hv52B0kRORlwYvrrQQRacSeLp1sxNjQIANHb24f1jdYqz/E2NCYVWw6HP7kJjlxVJUcZgD0n28DyvChcVoLw9igROAPA0/7Gu4sTEKS41492jFwAABbMTccnUaEXWGCLLn+8Y2DrlQHmr4sQNAOi0GkyNCUVViwWVzT0kcHxAS48NXVYHOK7/sKo0sshFRUwU9kfR1edAu0V5HVsDCXMr5KVHAwDSY8PFSr9KqzGk1FNVsPAUMyHuUgNKJIOE8aTxjtsS/h9TTaEw6rSKi9sC+udMY5cVPVZHkEfjO0jgBACjTotk90mKFp3JwWoMxUYITRJZvRgltjNQauBfsPjDR2fE+w4Xrygx7ImSS+8HCu+4rf7K10qM2wIAU6hOzCpU0oGKXFQBIiMuDA2dfahqsWB+ekywhyNbWI2hG4o+AeBtMlbaiZxZcC609cLmcEEfQueRiVJcasYLH58DICzmq7+RpUi3JkBVsH2Bp0tzQaawXnf1OQZVqFcSGXFhaO0RuorPcceqyR1aMQNEJhVu8xk8z6Oqtb8jtFJJiDAgTK+Fiwdq22jeTBR26r7t0lQAwkKuVLcm4GHBUWDp/UDC5sjnlW0AgC9rOxQrbgBlxvwFROC8/PLLyMrKgtFoRF5eHj799NNhr3333XexbNkyJCQkICoqCvn5+fjwww+9rtmyZQs4jht06+vr8/dHmTDpYjyFciZPsGjssqLP7oKGA6YouKEgx3GUgecDmFtznruAGfs/VaJbE/C24FCq+OQoLMgBc0YpOW4L8DiEK0gY+13gvPPOO1i7di1++ctfoqysDEuWLMGKFStQXV095PWffPIJli1bhh07duDIkSO47rrrcOutt6KsrMzruqioKNTX13vdjEbpZgwoUR0HC7bZT4kJVbzbhubN5FnnPnX3W/283ZpK6F3mSVpsKDQc0GNzornbFuzhyJrnd50Fk4hKjtsC+uMZKxS01vg9Bue5557D6tWr8cADDwAANm3ahA8//BB//OMfsXHjxkHXb9q0yevn3/72t3j//ffxr3/9C/Pnzxcf5zgOycnJfh27L6GMGN/BrGAZscp1TzFYIzyaN5OH/R8qfd4YQrRIjQ5FbVsvqlp6kBBpCPaQZElxqRlFu/oFTeH10xUbtwUoM3bLr8dfm82GI0eOYPny5V6PL1++HPv37x/Te7hcLnR1dSE21ruHU3d3NzIyMjB16lTccsstgyw8nlitVnR2dnrdAg0TOC09NnT1Uar4ZGCF79LjlFmTwhOy4PgONc6bCoVVpg0ULG7ruwvSAAhdxNcvn6nYuC0AyHLPmYudVlhsykgV96vAaW5uhtPpRFJSktfjSUlJaGhoGNN7/OEPf0BPTw9WrlwpPjZr1ixs2bIFH3zwAd5++20YjUZcddVVMJuHnnQbN26EyWQSb2lpaRP/UBMk0t2xFaDT+GSpFE/iyt+oyPLnG+xOFy609QLwdlEpFZo3k4PFbS2aFgeg3+qn1LitopKz+POBSkS7Gxh7zhs51/0JSADDwMq9Y63m+/bbb+PJJ5/EO++8g8TERPHxRYsW4fvf/z4uvfRSLFmyBH//+98xY8YMvPDCC0O+z4YNG9DR0SHeampqJveBJggtOr6hukX5GVQMdhKvbbPA4XQFeTTypa69Fw4XD32IBkmR0o3V8xVk+ZscYtxWy2CrnxLjtljdH2OIFkC/m0rudX/8GoMTHx8PrVY7yFrT2Ng4yKozkHfeeQerV6/GP/7xDyxdunTEazUaDa688sphLTgGgwEGQ/D90Jlx4Tha3S4GOxITo8rtalDDSTw5ygh9iAY2hwv1HX1IU4HVyh9UeVj9NDJdrMcDHaZ8Q5VKrMUDW5lUNFtEcSPn1Hi/WnD0ej3y8vJQUlLi9XhJSQkWL1487Ovefvtt3H///fjb3/6Gm2++edTfw/M8jh07hpSUlEmP2Z+IqeIKSsMLNB0Wu9juQql9YTzRaDhxcaXT+MRRiyhmLQayBvS/A+TtaggWNSqK2yosyMGiaUKs6/98eFr24gYIgItq/fr1+N///V+8/vrrOHXqFNatW4fq6mo8+OCDAAT30b333ite//bbb+Pee+/FH/7wByxatAgNDQ1oaGhAR0eHeM1TTz2FDz/8EOXl5Th27BhWr16NY8eOie8pVchsPHmY9Ss+woBwgzoKcff3FiJhPFGYWzNd4RlUzNXwr+N14Dih+m6bxS57V0OwYOuNGg5TALDyCiE+1cUDeq1G1uIGCECa+F133YWWlhY8/fTTqK+vR25uLnbs2IGMjAwAQH19vVdNnFdffRUOhwM/+clP8JOf/ER8/L777sOWLVsAAO3t7fjhD3+IhoYGmEwmzJ8/H5988gkWLFjg748zKchsPHnY/12mCk5UjEzR8kfCeKKIrgaFzxtPV0OkIQRdVgee/fA0tn5eI/vTeKDpsztxsdMKQB3xfgDweUWreN/mdKG41CzrOROQI/BDDz2Ehx56aMjnmGhh7NmzZ9T3KyoqQlFRkQ9GFljYH0lDZx/67E4Yddogj0h+qCnVt8h94s6IH2zBKS41w+niFRfs6C/UNG8GxlOQuJkYbM5EGkIQ484uUjLFpWZs/UJIwOE44BEF1P1RdhlYiRETpkOkUdCU7I+HGB9qKvLH3A3HqoVeOErJbAg0PM+Lf29KDxZlFBbkgE0PLafsFgP+wjODaixZv3KGrSnrluYgTK8FzwPfumyK7Ov+kMAJIBzH9cfhkLthQlSqxNUA9Nfc2Hb0AgAhUPb5XWcVEfwXSJq6rbDYnNBwwNQY5c8bQNiwWKkWJ6/sFgP+QjxMqWCtYXV/Hl06o7//XatF9nV/1BGlKSHS48Lw1YUOisOZINVD1KVQMoUFOXC5eGwqNcPmcKFol5nEzThhcybFpPzeZUD/afyWeSn4f1/VI9VklL2rIRiIbk0VWIs9Xd3psWE43dAl/t3Iec4o/69dIrD0TTFg1KMWDqVvjo0+uxMNnULH+EyVBP0BwFqPxUfpHY39gVoCjAF41S750TXZAACbk5e9qyEYqGneeMI+rxLCKEjgBAgWT2G+2A2g/4+H4inGTo3Kgv4YnpuS0jsa+4MqFVW+Zq6GwoIc0crZ3G3FA0uyZO1qCAZqi9tiiC4qBXgZyEUVIAZmNlS29CiiUmQgqVRR0B+DzZFLp5pwvLYDCzJjyN0wTtRS5A/wdjWYQnWIDtOh3WJHtTueghgbTheP2jZ1ucMZ6e6DQLUCKu6TBSeAFBbk4IdXTwMA1LT2krgZJ2oK+gO83Q23XpoKAEiMMpK7YZyopdz+UCjpNB5I6tp7YXfy0Gk5pJhCgz2cgML+TqpbLWIVbLlCAifAbFgxS7yv01I8xXgQTcYqcDUA3u6GNI9FR+6ZDYFGTTVwBsIETo0C4ikCCZszaTFhqgsfSI0OhYYD+uwuNHVZgz2cSUEuqgDzwsfnxPt2Jy/7SpGBRG0ncU93w8DAP5ozY6Orz47WHhsA9QhjT8iCMzGG6iKuFvQhGqRGh6K2rRdVrRYkRhmDPaQJQxacAMJcDtPclWmvn5VIroZxwFxUalx02EbVbrGjo9ce5NHIB7ZRxYXrEaGS3mWeiO1hyIIzLliWq1oOUwNRSlshEjgBwjOe4pqZCQCA7IRwiqcYIw6nC7VtvQDUeRIP04cgPsIAgNwN40HN7ikAomuT5sz46K+3pb61Buiv/SP3VHESOAHCM54ig+IpxkSRh/Cr7+iDw8VDH6JBSpRRlbWDlHKqCiRqc2sOhB0GatsstMaMA7XPG2Yxrm6RdyaV+my2QcKrUuSAjYriKYaG1Q4CgMvTYwAAaTGheHH3OdEapibSY8NwpKpN9qeqQMJSXdV6Ek+OMkKv1cDmdKG+o1c1rSomg1fvMpVa/pTi2iSBEwSY+a/GnYanlpou48WzdlDBrEQAgIuHatPrxVOVAupTBAp2iMhU6Ual1XCYGhOK8uYeVLdYSOCMgdYeG7qtDgD9Lj610W/BkbfAIRdVEJgaEwqOA3psTrS4MzyIoWEuvNLTjQCAiuYeVYobwFPgyHvRCSRqLbfvSbpCTuOBgv0/JUcZYdRpgzya4MDmTIuH2JMjJHCCgFGnRbI79Y42q9EpLMgBM3JpVdyLiWJwxgaL3bI6nKjrEALTmdVUjbFbJIzHh9oa+g5FlFEntsORsxWHBE6QSFOICTAQFJeawQpqOlXci4ltVHXtvbA5XEEejXRhsVsbd5wGzwNhei3iI/Sq7fumFHdDoFB7gDFDCS0bSOAEiQw6VY0JtinptcJUvX9xhmrT6hMiDTDqNHDxgsghhoa5NbfsrwQgbPAvfHyOYrdorRkTYg0cFVtwAGXMGxI4QYIqjI4OEzcPXjMNNqdgsXh8xWzV1g7iOK5/3sh40QkEhQU5uM5db+rMxS7VihugP1W8SuYpv/7EsyTFwBo4anRrAv2HcDnvUSRwggTz71IBruFhtYNumJsMoD/oT821g5RSgCsQZMVHAAB4HtBrNaoUN0D/Yaqzz4F2CyU1DAVzaxaXmvu7z8eGqdatCfTvUXJeayhNPEgowfznb1jtoPePXQDgHfSn9s1K7gW4AsGeM0LmnVbDweZ0qbbvW6hei4RIA5q6rKhutSA6TB/sIUkOz5IUjA9PNuDlPedVa/lTgpeBLDhBgpmNGzr70Gd3Bnk00kY0Gas86A8Y3HSTGJriUjPKmwUR+Mb9V6rWrclQgrvB3xQW5ODe/AzxZzWLG6B/rbnQ3guHU55JDSRwgkRMmE5s/lfbRovOSIj9hEjgDKqCTQyGuRVC3G6F9Ngw0a2pVpFDFuOx8Y3p8eJ9Nbs1ASAp0gh9iAZOF4+69r5gD2dCkMAJEl4Bo7RZjUiVysume8LmDKuCTQzG6eLxo6unweHioeGA1OhQAFB37FYcpYqPhbcOVQEANBxEt6Za0Wg8kxrk6RKnGJwgkh4bhq/rO+lUNQosEFutZdM9GVgFm3UYJ/pZt2wGDle24tVPypEaHQp9SP85Tq0ncrlvVIGguNSMvWebAQA/uiYboTqtGJOj5nlzrrFbtnsUCZwgQpVpR6fP7kRDp2AeVXvhLQAwhGiREmVEXUcfqlstJHCGgdya3mSIWZtUP2komFszMy4MlS0WpMeG4bsL0gFA1SJH7kUiSeAEkbRYShUfjdq2XvA8EK7XIjacsj8Awd1Q19GH6haL2GWd8IZ6UHnDygvUdfTC6nDCEKLOHkvDwUpSvMcyNt1rMxM1anNrFrlT44c6hBeXmuF08WKWq5ShGJwgopSW9P7E0z1FXdcFKGB0dMit6U18hB5hei14HrjQRlacgaxbNgM/uW46altZ7zLvkhRy2Mx9CasLdLymHUD/HiW3ukBkwQkingGjLhcPjUwmTSBh1VfpJN5Pf2VaEjjDUUUuKi9YUsPphi5UtVowLSEi2EOSHBc7+2BzuhCi4ZBiMgZ7OEFlYF2gmlYLnt91FkW7zLJKnScLThBJjQ6FVsPB6nChscsa7OFIkuohTlRqh1ybo1MtVqMND/JIpAM1+B0ZdmCYGhOKEC1tjYUFOSi8fjoAoNvqkJ24AUjgBBWdVoPUaOGkQO6GoWGdbFlfGMKjaBtlxAxJr82JJveBgYRxf5+loRr8qrXP0lCQW3Mw65fPFO+HaDhZiRsgQALn5ZdfRlZWFoxGI/Ly8vDpp5+OeP3evXuRl5cHo9GIadOm4ZVXXhl0zbZt2zBnzhwYDAbMmTMH27dv99fw/Qo7YVIjvKGhbJjBsP+Li51WqoI9BGzOmEJ1MIXpgjya4MPiKcqbuwH0WyrkFk/hb2itGYxnHSCHi5ddXSC/C5x33nkHa9euxS9/+UuUlZVhyZIlWLFiBaqrq4e8vqKiAjfddBOWLFmCsrIy/OIXv0BhYSG2bdsmXnPgwAHcddddWLVqFY4fP45Vq1Zh5cqVOHTokL8/js8hd8Pw8DxPi84QRIfpEGkUwudo3gyG5ow3rMDhx6ebAAhzhokbubkc/AkVFPWGzZG5qVEAgMXZcbKrBO53gfPcc89h9erVeOCBBzB79mxs2rQJaWlp+OMf/zjk9a+88grS09OxadMmzJ49Gw888AB+8IMf4Pe//714zaZNm7Bs2TJs2LABs2bNwoYNG1BQUIBNmzb5++P4HMqkGp6mLiv67C5oOGCKuxqt2ikqOYsXPj43ZBVscjcIkMAZTGFBDv7rqkwAwJmLXSRuhoDmTT+eAvjGuckAhJhRubU78avAsdlsOHLkCJYvX+71+PLly7F///4hX3PgwIFB199www04fPgw7Hb7iNcM955WqxWdnZ1eN6lAKb/Dw/5PUkze1WjVDHM32BxC87tqmaZv+hPWaT2dTuJebFgxW7yv08ovnsLfiPOGAtPFukCFBTn9bT5aLbJrd+LXNPHm5mY4nU4kJSV5PZ6UlISGhoYhX9PQ0DDk9Q6HA83NzUhJSRn2muHec+PGjXjqqacm8Un8h9wrRfoTKtY2mIHpm9XkbhgEncSH5pW958X7dqcQT0HzRaCzz442i3CATosla7Fn3Z+Be5Sc5kxAjsUDC7TxPD9i0bahrh/4+Hjec8OGDejo6BBvNTU14xq/P2HquKXHhm6rI8ijkRa0UQ1NYUEOls5OBAD8+UAliZsBiLEUNG9EmAhOixE27xvnJsvK1eBvWCxbbLgekUYKTPeErb8NnX2yS2rwqwUnPj4eWq12kGWlsbFxkAWGkZycPOT1ISEhiIuLG/Ga4d7TYDDAYJBezx5WDjs6TId2ix3VLRbMcQd0yakctr+gtM3h+a+rsrDrVCN4HtBrNSRu3LhcvFiNluaNgKeFr669F1u/qMHM5EjMSY1SdZ8lT5h1gg5Tg4kN1yPCEIJuqwO1bRZMT4wM9pDGjF8tOHq9Hnl5eSgpKfF6vKSkBIsXLx7yNfn5+YOu/+ijj3DFFVdAp9ONeM1w7ylVWDyF0R1fQvEU3lBWw/B8fLpRvG9zuugk7uZiF1WjHYhnPEVarHzjKfwJWYuHh+M4r3kjJ/zeqmH9+vVYtWoVrrjiCuTn5+O1115DdXU1HnzwQQCC++jChQv4y1/+AgB48MEH8eKLL2L9+vVYs2YNDhw4gM2bN+Ptt98W3/PRRx/F1VdfjWeeeQbf/OY38f7772PXrl3Yt2+fvz+OTxmqHDbFU/RDi87QFJeasXlfBTQc4OKBNUuy6CTuhqrRDsbTCpwR571RqX2+MOgwNTIZsWE4Vd8pu/Ywfhc4d911F1paWvD000+jvr4eubm52LFjBzIyMgAA9fX1XjVxsrKysGPHDqxbtw4vvfQSUlNTUVxcjNtvv128ZvHixdi6dSt+9atf4de//jWys7PxzjvvYOHChf7+OD6nsCAH+8414fOKNmz8zym4eJC4AWCxOaga7RB4CuBtR2tR1WJBwewkRBp1JHLQv3GTe2pohiovQJA7fDTS48iCMywPPfQQHnrooSGf27Jly6DHrrnmGhw9enTE97zjjjtwxx13+GJ4Qef2y6fi84o2uCieQqTGHUcRZQxBdJg+yKORDp7uhi8qW1HVYhHdDex5NVNNmXcjwiqnN3dbYbE5EKanfssAWYtHQ67ZvjS7JcDhyjbxPounULvIERcc2qi8GOhu+NQsz/RNf0Eb1ciYwnQwherQ0WtHdasFs5Kjgj2koONwunChTThQkTAeGrnWayMndZApLjXjH0dqAQjNzNYW5FD6Jvp7c1E36OGR66LjT0jgjI5cT+P+or6jDw4XD32IBkmRFJg+FJ6xW6xsixwggRNEWDzF2qU50Gs1cLh43J43VXblsP0B+cRHh1VcpTYf/fQLHBLGwyHXeAp/weKR0mJCoVF55upwpEaHQqvhYHW40OiOjZQDJHCCCIunWLt0Bqa6q2dWtag3fbPIQ9RVDTiJU5+lwfSfxKkTPQB09dnR2mMDQK7NkSDLnzdk9RsdnVaD1GjBuiWneUMxOEHEK54iNgzlTT2qTt9kdYGA/j+ijLgwr8whoh+2ibdZ7OjssyNK5RVY2ZyJcxcmI4YmgzKpvCCBMzbSY8NQ09qLqhYLrsyMDfZwxgStAhIhIy4cQBOqWtV7GvesC6R1t934+HQjNu+roNT5IYgwhCA+Qo/mbhuqWyzInWIK9pCCCrk1xwbbyGtkdBL3J9WtrDkruTVHIj02HJ+hRVYWHHJRSYQ0CvwDIIicNUuy4HQHspG4GRm5Vhj1FZ5uzYEncXJrDg2z/NW0WVTnBh8KsuCMDTm6xEngSIQMlW9Unlw/q7+nGNUFGhm1zxvm1iwuNXt1n6d2J8OTYgqFTsvB7uRR39Eb7OEEFZ7nveYNMTwDq2DLAXJRSQRx8rRYRu22rnRe/6wcAMBxVBdoNNRemdbTrcnE3pmGLnz09UWy/A2DVsNhakwYKpqFmL+pMerd2Dt67ejqcwAA0lT8/zAW5BicTgJHIjBXQ5fVgTaLHbHh6qzeW1xqRsnXQiPJ7y1MR2KkkVoQjACLG1BzPMXAnm4kbkYnLdYtcFosWJwd7NEED7ZZJ0QaEKrXBnk00oa5Npu7beixOhAug0B+clFJBKNOi6QoAwB5KWRfwtwKM5IiAAhF/ljKvNrrAg0Hs/ypOTgdAB66tn+X1mk5EjejoGbXplc5CuaeoritUYky6hAdJmRq1rTJY96QwJEQrGpvlYyCuHwJqwtkCBFOUuzEoNa6QGOBmY3r2vtgd7qCPJrgsfE/p8X7didPYngURNemCgWOZ9yWZ4AxxW2NjtxKDEjfxqQi0mLD8Hllq2ozqVhdoP/9VIjB8Qz6oxP50CRGGmAI0cDqcKGuvdddbkBdFJeasXlfBQBgemIEbrs0ldyaoyBmUqlQ4Hi6NHNThV5c9R29eLfsArk2RyEtNgzHaztkM29I4EiIfneDPCaPP2i32NDpDvqjtM3R4TgO6bFhMDd2o6rFojqBw07dBbMSUXq6ERmxYYNicmjDGgwFp3vPkQPlrSRuxoDc5g25qCSEHNPwfA37w0mINCBMT/p7LKh53jC3ZnaiELdFbs2xwTaqjl47Oiz2II8mOHiKmRANxW2NBbmtNSRwJAQV++u3XmWQ9WbMsMaScll0fMk696m7v/u8t1tzHbX3GJJwQwjiI9Sd1PBcyRnxvsNFcVtjQW6FRUngSAi2ODd09qHP7gzyaIIDq5JJzRLHTrrYqFWdwemARzaMylx0k4HNG7lsVr6kuNSM4tJzAIAwvRbrluZQpuYYYH9ftTKpgk0CR0LEejQJrJVJGp6v6U/bpI1qrLBFp7pVnVVpeZ7vz4YhYTxm2LxRW4kBFrf1rfmpAAR33aNLZ1A5ijGQHGWUVRVsCnKQEBzHIS02DKfqO1HVYsH0xMhgDyngVHl0ESfGRppHjxg1VsFu7rbBYnOC44CpMaHBHo5sYPNGLhkxvoLFbUUaQ/BeWR0y3UKPxeDIwTIRDIrcKfRpMWEoH1AFu7jUDKeLl5xLmCw4EkNudQZ8TQ2dxMdNWmwoOA7osTnR2mML9nACDusGnWoKFWsoEaOj1rWmP25r8GGK4raGh9UPcrkbIbNYUSnXDyKBIzHkFqXuS/rsTjR09gGgIOPxYAjRIiXKCECdJQaoWeLEYIcItQkcRhXF+40LlplY6Z4v1a0WUdxINcWeBI7ESFexwKlts4DngQhDiGp7cU0UNWfgVZLAmRDsEFHf0QubQ31VsPszNineb6wUFuTgmhnxAIBX9p6XtLgBSOBIjv5CSuoK/AP6T5LpsWGqiyOZLGq2/ImZd7RRjQnWiykh0gCjTgMXD1xoFwJG1dKLyeniUesOyidhPD6+tzADAODiAb1WI1lxA5DAkRzsNFHT1guXyoLdyNUwceRWYdSXUGD6+GCxFC98fE6cN57uBinGUviahs4+2Jwu6LQcUqMpMH087DvXLN63OV2SzjqjLCqJkRpthFbDweZw4WJXH1JM6vnjo1TfiZPuzgRRW0YM0O+Wo9YeY8OzTcG0eGHe/Hl/BT4+3SRpd4MvqWoWrH5TY8JUIeh8RXGpGX85UCX+/ONrsiXdEoUEjsQI0WowJToU1a0WVLVYVCVw+qvRkqthrLDUzWtmJADwrmki1dRNX9JtdaDFnTlGFpyxM7AXk5rEDdBv9SNRPHY8A4r/dqgaDZ19WD43CaF6rWRFDrmoJIgYT6EydwO5GsYPczfsPNEAALjYaUWf3akadwMTxbHhekQadUEejbwoLMgR5wfHSW9z8ifMlZtJa82YYfWDCgtyvGL+pNz3jSw4EiRdZv0+fIFn0B+dqsaO50lcH6KBzeHCxh2n8OcDVao4kZN7auIwCx8A8Lzws9LnC4PVTkqn1h5jxtMSnBkXjkMVrahsFv7+pDpvSOBICOZuEANGPQSO0t0NFPQ3cQa6G9QibgCy+k0UZuH7wVWZeP2zStESCEh3s/IlbGOmelsTo7+GkrSzfclFJSHYInPiQgeA/vRXNbgb2B8KBf1NjMKCHLD/Nq2GU8UmBXj2LqONaqx4xlL84qbZCNFwcLp4rFmSpYpeTJ69y0gYTwzW3qJS4gKHLDgSYuBJXA6VIn0FuRomR3GpGcwF7nTxqnE3iIHp5GoYM56xFIDQv6uyxYLrZyUh0qiTZCyFL2ntsaHb6gDH9RfIJMaHXOpu+dWC09bWhlWrVsFkMsFkMmHVqlVob28f9nq73Y6f//znmDdvHsLDw5Gamop7770XdXV1Xtdde+214DjO63b33Xf786MEjMKCHDx83XQAQJvFrgpxA5CrYTIwEbx8ThIAwZqhhpM4QLWTJsK6AeuJ2FW8pUcVvZjYWpMcZYRRR73LJgL7e2vutqGrzx7k0QyPXwXOPffcg2PHjmHnzp3YuXMnjh07hlWrVg17vcViwdGjR/HrX/8aR48exbvvvouzZ8/itttuG3TtmjVrUF9fL95effVVf36UgPLYDTPF+yEqcTeQBWdieFr4Vn8jCwDAA1i/bIbiRY7N4UJ9hzswnQTOhGGbVaVKsjbFHlS01kyYSKMOce52OlIuLuo3F9WpU6ewc+dOHDx4EAsXLgQA/OlPf0J+fj7OnDmDmTNnDnqNyWRCSUmJ12MvvPACFixYgOrqaqSnp4uPh4WFITk52V/DDyqem5JDJe4GVr+FXA3jw9Pd0OhuVFrbZsGD12SLzyuV2jYLXDwQptciIcIQ7OHIFk8Ljhogq59vyIgLQ0uPDdWtFuROMQV7OEPiNwvOgQMHYDKZRHEDAIsWLYLJZML+/fvH/D4dHR3gOA7R0dFej7/11luIj4/H3Llz8dhjj6Grq8tXQw8q7EQ+OyUSAHBVdpziT+I8z9OiM0E83Q0JkQaE6rRibyGluxs8i7VR77KJk6kyC061uNbQYWoyZMgg0NhvFpyGhgYkJiYOejwxMRENDQ1jeo++vj48/vjjuOeeexAVFSU+/r3vfQ9ZWVlITk7GiRMnsGHDBhw/fnyQ9YdhtVphtVrFnzs7O8f5aQKDp7uBA3Cqvgsp0aGiuwFQZgpnu8WOrj4HADIbTwaO45ARF4bTDV2obOlBVryyF3Bya/oGtlFVt/SA53nFi0WK9/MN7P+vqlm6wnjcFpwnn3xyUIDvwNvhw4cBYMg/lLH+Adntdtx9991wuVx4+eWXvZ5bs2YNli5ditzcXNx999345z//iV27duHo0aNDvtfGjRvFQGeTyYS0tLTxfuyA4FUp0r05VTb3SLpS5EQp8rBKsQUnKcoAo06rmo7G/kBM32yW7qnKV5DVzzekxYaC44AemxPN3bZgD8fvUEsY3yCHVPFxW3AefvjhUTOWMjMz8eWXX+LixYuDnmtqakJSUtKIr7fb7Vi5ciUqKirw8ccfe1lvhuLyyy+HTqeD2WzG5ZdfPuj5DRs2YP369eLPnZ2dkhQ5nu6ELHHySLtS5ETxLCzGNqiM2HAvKxYxfjLi1dNVnKrR+gZDiBapplBcaO9FVUsPEiKVG8/UbXWIIo4C0yeHHFLFxy1w4uPjER8fP+p1+fn56OjowOeff44FCxYAAA4dOoSOjg4sXrx42NcxcWM2m7F7927ExcWN+rtOnjwJu92OlJSUIZ83GAwwGOT1R5supuFZ0dVnV1yfHc+aP4uzhe+42+ZQTVq8v5DDqcpXVFKRP5+RGR+GC+29qGyx4IrM2GAPx28wt2ZMmA6mUGWtqYGGuTbrO/rQZ3dKMuXeb0HGs2fPxo033og1a9bg4MGDOHjwINasWYNbbrnFK4Nq1qxZ2L59OwDA4XDgjjvuwOHDh/HWW2/B6XSioaEBDQ0NsNkE1X3+/Hk8/fTTOHz4MCorK7Fjxw7ceeedmD9/Pq666ip/fZyAYwrVIVYGaXiTgbne9p9vAQB8XddJ4maSiH5xhc4ZhsvVX402kyw4k0YtmVRk9fMdMWE6RBoFG4lUrTh+rYPz1ltvYd68eVi+fDmWL1+OSy65BH/961+9rjlz5gw6OoTWBLW1tfjggw9QW1uLyy67DCkpKeKNZV7p9XqUlpbihhtuwMyZM1FYWIjly5dj165d0GqlpyAnQ6YKNqvCghywiCw1tRjwFyywuKbVAofTFeTR+B4Wu3Wxqw82hwshGg6p0UYAoNitSaCWTCqy+vkOjuPEw4VU9yi/tmqIjY3Fm2++OeI1PN8fOJuZmen181CkpaVh7969Phmf1MmMC8fR6nZFuxuKS81g37iaWgz4i6RIIwwhGlgdLtS19ykuzoDFbtW0CQvqlJhQhGg1FLs1SdRiwaHAdN+SHheGry50SHbeUC8qCZMZr+yMGLYpMX5yXbai0+EDgUYjpIqfvdiNipYexQmcgf3a0mPDVNOvzZ+wk3hFs7JTxaupoKhP6bf8SXOPom7iEiZD4pNnMrBN6XsLherUceF6/PcNs1TRYsDfKP00XliQgwVZMQCAz841k7jxAayWUFefA+0W6fYWmgheJSkGWHDIrTk5MtTsoiImB4unUKJfnNX8yYoPx1uHqkVrFduklFTzJ9CIpyoJF+CaLAkRQtyNiwf0Wg2Jm0kSqtciOcqIhs4+VLb0IMad4KAEmFvT6eJR1y70LssYYPkjJgaLZSKBQ4wbVoiqqcuKHqsD4QblfF2s5s8L7pOVZyYMbVaTQ+kWHAA4VNEKQGhGa3O6KHbLB2TEhaGhsw9VLRbMT48J9nB8xkC3ZqhOi7c/r0bRLjNZ/iYJO5jWtllgc7igD5GWU0haoyG8MIXpEBMm1GpQopsKACrcnysrXlmxIsFE6bVwnt91Fs3dQuuVj9ZdTW5NH6HkeVNYkINvz58CAOizO0nc+IjESAOMOo3Y/05qkMCROEwhS9UEOFlYAHWmwvsmBZJMt1isae1VnKuvuNSMol2CkNFqOKTFhon1lEjkTA6lV8G+ZKrQ8ZoHuTV9hXequPSEMQkciaPkUxXQH19Exdp8R4opFHqtBjanS4w5UApOF48786YCANJiQqHTCkuYEvu1BRqlrzXvH7sAANBwEN2axORJl3AcjnKCOhSKkpsndvTa0dojVKhWeufrQCJYNkJxvqkHVS0WpCmoqNm6ZTPw9ufV+MeR2kFWPzqRTw4pb1STpbjUjGM1QkHZ333nEjR09lFJCh8hljORoDAmgSNxmLtBiZlUTLQlRhoUFUAtBTLjwnG+qQeVLT34Rs7ovePkRAVza5LVz6ew1OnWHhs6++yIUkj/O5YtFWUMQWefA1kJ4Vh5pdBsmUTO5JFyexjaVSROhoItOEzxU/yN71FyJhUTONMSaN74kkijDvERejR321DdYkHuFFOwh+QTnC4ehddPxwu7zwHoF8ZUkmJyFJWchVbD4XJ3xp3nWlNcaobTxYvZssGCYnAkTpb7j7GxywqLzRHk0fgWtlFl0Unc5yjZ8kcWHP+RocA4nHXLZuCWS1PB80CkIQTxEf01fgoLcoK+CcsVVl9o95lGAP1JDcxiptUEvxo2WXAkjilMh+gwHdotdlS1WDA7JSrYQ/IZlEHlP5RqwXG6eFS7RRvFbfmejLgwHKlqk6S7YTKIh6mEcMW2oQg0nvWFWOD2xh2n8L/7KiSTgk8CRwZkxoXjmKUdlc09ihI4FeJGpZwgWKng2Yne5eKhkcBpyhfUtffC5nRBr9UgNTo02MNRDMzdMFRSg1TcDZNBFDgkin3KwCKKUhI3ALmoZEF/QzPlnKp4nkdFUzcAsuD4gynRoQjRcLA6XGjo7Av2cHwG26jS48IkYQJXCszdcLq+E0B/wKiU3A2ToaKJ3Jr+orAgB8woptVwkhE3AAkcWaDEruJtFjs6+4SYItaSgvAdIVqNmB6upHiKyhY6ifsDVkdox4kGAML/s5K6tFNguv8oLjWDd8dpsxgcqUACRwYosQAXW3BSTEaE6rVBHo2yYN2Th0rflHv35PImEjj+orAgBw9dmw1ASGpQirgBPFvC0LzxJUwEL52dCEDwNkipojgJHBkg5ToDE6WSMmH8BnM3dFjsAPqFsRLcDWTB8S8/u3GWeD9EYu6GidLVZ0dTl9C7jNzhvsPTwvfAkmkAABcPSbVNoSBjGcAW84bOPvTanIqweFANHP8xMPCvqtmiGHcDpYj7F89NyeF2N8h5vgBAZbNwMIyP0CumeKEUcLp4cT1pdMf51bZZ8KNrponPBxsSOBKHZTeYQnXo6LWjqrUHs5KFTCo5Zzf0ZzVQBpU/KCzIQUVzD7aXXcCHJxuw82SD7MWN3elCbZvQW4tiKXwPE8FzU6Nwsq4T+dNiFVHpt7xZSGYgq59v8dx3EiINiDCEoNvqQHWLRTLzhVxUEoe5G0J1gtWGnUbk7m4QLTh0EvcbbJFRSvfkmlYLnC4eYXotEiMNwR6OovC08N1ySSoAICHSKCl3w0RhayYJHP/BcZx46DjfJJ1YURI4EodlN7BUXyVkN/A8T4tOAHi/7IJ4Xwndk5nVLyOOirX5Gk93A9uoypu7FdGlvUK04EQEeSTKhq3lzGImBchFJQMKC3Kw/3wzDpa34tmdp8VALjmKGwBo7rah2+oAxwn1TAjfU1xqxqZSM2LD9WjtseE7l0+RvbtBTPUlUexzPN0N2W6BU9HUA57nZTtfGOQODwzT3AKygiw4xHi5y9391sXL393A3FNTokNhCJF/wLTU8LTwXZkpNMKbN8Uke3eDGGBMG5VfSY8Nh1bDocfmxMVOa7CHMyl4nke5KHDIguNP+i1/JHCIcfJFZZt4X+7uBiqb7l+83Q3Con6+Sf7uhv4Ucdqo/Ik+RIO0GKENRnmTdNwNE6G1x4auPsFanEHWYr8iuqgkNGfIRSUDikvN+NuhavHnh67NlrW7gWrg+Bdvd4MgBliBPDnOF0ZFE7kaAsW0hAhUtlhwvrkHi6fHB3s4E4YdplJNoTDqyFrsT5gFp81iR1uPDTHh+lFe4X/IgiNxPN0NyVFGAMDSOUmydjdQDZzA0Z/ZIJ1T1UTosztR1yEE2pMFx/9Mk+BpfCKUk7U4YITpQ5BiEvYoqbipSOBIHE93Q3aie7NqlJ+7ochDjFU0e3cRl3v7ACmT7RYDFzut6LY6gjyaicOqeEcZQxATRsXa/M20AZY/uULu8MAiNTcVuagkzkB3w2fnWsQ6A3JyN7B6PkKKeL+LytNCRfgeU5gO8RF6NHfbUN7UjUumRgd7SBOiwqNYG6WI+x/PVHE5U0kCJ6BMSwjH/vMtorAMNiRwZES2R8Co3BjYPkCr4fD+sTo8X2qWdcq7HJiWEIHm7laUN/XIWOBQ3aRAwgRObVsv+uxO2caviBYcqnwdEFiquFQsf+SikhFyFjiAIHLuzJsKQHC9kbgJDNkyjcPxdmsKY2dxW+TW9C8JEQZEGkLA8/Jt8uty8VQ7KcBkSczyRwJHRrAYnOoWC+xOV5BHMzEuTYsW78u9no9cGJhJJReYW7O41OxV+VrubUrkAMdxmJbI5o00NqvxUt/ZB6vDhRANhynRocEejipgMX+VLRZJxIf6VeC0tbVh1apVMJlMMJlMWLVqFdrb20d8zf333w+O47xuixYt8rrGarXikUceQXx8PMLDw3HbbbehtrbWj59EGiRHGRGm18Lh4mV7qtp2RPieNJz86/nIBblmUrFA+udKzuJEXQcA4FBFq6zblMiJ7Hh5zhsGKyuQHheGEC2d5QPBlJhQ6LUa2Bwu1LX3Bns4/hU499xzD44dO4adO3di586dOHbsGFatWjXq62688UbU19eLtx07dng9v3btWmzfvh1bt27Fvn370N3djVtuuQVOp9NfH0UScBwnazdVcakZZTXtAIDffecSWae6ywk2ZyqaeyRxqhoPhQU5+Ml12bDYhL/tvx2qJnETIMRAYxlZ/rzcmi3e7ilya/ofrYYTCypKYY/yW5DxqVOnsHPnThw8eBALFy4EAPzpT39Cfn4+zpw5g5kzZw77WoPBgOTk5CGf6+jowObNm/HXv/4VS5cuBQC8+eabSEtLw65du3DDDTf4/sNIiOyEcHx1oUMSk2c8MLdChCEE3VYHshMjsNLdfkLORQvlwNSYMOi1Gljdp6q0WHkVyls2Jxkv7T4PgNyagUSsgi2RjJixwNyaANBusQPwdmtStqb/mZYQDnNjNyqae3Dt8Nt8QPCbBefAgQMwmUyiuAGARYsWwWQyYf/+/SO+ds+ePUhMTMSMGTOwZs0aNDY2is8dOXIEdrsdy5cvFx9LTU1Fbm7uqO+rBEQLTqN8Fh1ACCr+yXXZYi2W6W7/vtzq+cgRrYYT+zfJTRgDwMu7zwEAOJBbM5D0W3C6wfPy+Pv0dGvuPn0RgFB0jtyagUNKNZT8ZsFpaGhAYmLioMcTExPR0NAw7OtWrFiBO++8ExkZGaioqMCvf/1rXH/99Thy5AgMBgMaGhqg1+sRExPj9bqkpKRh39dqtcJq7W8a19nZOcFPFXyyE+Xpolq3bAbKqtvw0u7zSIw0wBTaX6yNFh3/k50QgbMXu3G+KfinqvFQXGrGR18LG9W9+RmIizCQxS9AZMaFg+OArj4HmrttSIg0BHtIY2JgSYrSU40kbgKIWOxPAplU47bgPPnkk4OCgAfeDh8+DABDFuTieX7EQl133XUXbr75ZuTm5uLWW2/Ff/7zH5w9exb//ve/RxzXSO+7ceNGMdDZZDIhLS1tHJ9YWnjG4MjlVMU41yhMeGa9IQKH52lcLjC3AouhmJ4U6XVCJ0uOfzHqtGL2kZzmDQD88Opp4n2dliNxE0BYWYoKCVhwxi1wHn74YZw6dWrEW25uLpKTk3Hx4sVBr29qakJSUtKYf19KSgoyMjJgNguLWXJyMmw2G9ra2ryua2xsHPZ9N2zYgI6ODvFWU1Mzjk8sLTLiwqBxn6qauq2jv0BCnHMvkjkkcAKOHIPTWZsSh9t9mUNuzYAjuhtkFIcDAP/336fE+3YnT2I4gLBif3UdfbDYgtseZtwuqvj4eMTHj95dNj8/Hx0dHfj888+xYMECAMChQ4fQ0dGBxYsXj/n3tbS0oKamBikpKQCAvLw86HQ6lJSUYOXKlQCA+vp6nDhxAs8+++yQ72EwGGAwyMO8OhpGnRZpsWGoarHgfGMPEiONwR7SmDlPFpygISW/+FhZt2wGem1OFO0SXA2ewphO5IFhWnw4PjnbJCsLTnGpGW8erAIAXJERg6tnJJBbM4DEhOsRHaZDu8WOiuYezE01BW0sfgsynj17Nm688UasWbMGBw8exMGDB7FmzRrccsstXhlUs2bNwvbt2wEA3d3deOyxx3DgwAFUVlZiz549uPXWWxEfH49vf/vbAACTyYTVq1fjpz/9KUpLS1FWVobvf//7mDdvnphVpXTkeBoHALNb4GSTwAk4zEXV2GVFV589yKMZO4IrFogN1yMuQhmHFDmRLbNUcebWXJgVCwDIIbdmQGFp+syt7NmTKhhp+n6tg/PWW29h3rx5WL58OZYvX45LLrkEf/3rX72uOXPmDDo6hCJeWq0WX331Fb75zW9ixowZuO+++zBjxgwcOHAAkZGR4muKiorwrW99CytXrsRVV12FsLAw/Otf/4JWK89+KeNFjqX3++xO1LQKxQnJghN4oow6MUhULpsV4BG3lUBzJhjIzUXF3JrR7o7z5NYMLCxNv88uVNpna02wqo/7tdlmbGws3nzzzRGv8QyUDQ0NxYcffjjq+xqNRrzwwgt44YUXJj1GOdJvwZHHogMISt7FA6ZQHRLoJB4UshPC0dRlxfmmbq+WGVLG3NgFAJieRAInGDDLX3WrBTaHC/oQaVcEXueuc3P97/cAAHKSyK0ZSAZmsJU3dXvVIAr0d0DdxGWImCreKB8Ljtkj/makLDrCf0xLiMDB8lZZWXDMFykwPVgUlZyFhgPC9FpYbE5Ut1pE62txqRlOFy8KCilhdThR6a5iPCMpcpSrCV9TWJAD88Uu/OvLerx/rA48ELQ0fWnLcWJImAWnrqMXvTZ5tKcgV0PwkWPsVn/mHW1UgUar4VC0y4xwg3AOZoHGUm92Wt4kWIsjjSFIlEntHqXx8PWCmOER3OrjJHBkSGy4HjFhOvC8dxCXlKEMquAjt95CVodTbCqbQy6qgMPiVpq6hHIU5c09QXU3jBVmLZ6RFEnW4iDxnxP1AARxE8zq4+SikinZCRE4XNWG803dmJMaFezhjIpowaGNKuAUuU/b354/BYDQhNDp4qHVcJJ2NVQ2W+B08XQSDyKFBTnYf74ZB8tb8ezO03DxwXM3jBXzRSFui9yawaG41IxNu8ziPGGiGAh8HBQJHJniKXCkjsPpEst2k4sq8LDMBhfPwxAiNN280NaL945dkHQDQjHAmOK2gsqqRZk4WN4KFy+PZqdi3BbF3wScoSx8AwOPAzl/SODIDHYaz05kqeLedQakeBqvbrXA7uQR6lH6nQgcngtMfIQe1m4binadxfayC5I+jVOAsTT4orJFvM/cDVKdM0C/MKZ5E3hYmv7A+cF+DnSaPgkcmcFO49+6LBVAf2yLp3KWGsw9NS0hHBqJBiYqnYGnKKmLG6B/3lCAcfAoLjVjy/4qaDUcnC4eP7gqU9JVgYUMKiFuizKoAs9Ih2vKoiJGhQX+vXesDoDQsfX5XWclHfhHPaikQWFBDrRuV4+Gk+YG5QnFbQUXz0PTTLdYWDgtTtJVgcW4LUMIkqIobkvtkMCRIYUFOVjr3pz67C4UeQR0SZFzFymDSgoUl5rhdBfWdPGQ5AbF8IzbImEcHDzdDbOSBYFzpqFL0lWBz7IA4ySK2yLIRSVb1i6bgedLzeABhGg4yYoboN+CQwIneLDT+A+uysTrn1WKrk5AmpacKnfcVphei1QTxW0FA093w0wPgQNIc84A/Sni5NYkALLgyJZit7gBAIeLl+xpnOd5qoETZDxdDb+6eQ7C9Fo4XTzuX5wpWVcDCzDOToiguC0JIAoct4VEqpg9LDgEQQJHhrAN66rpcQCExUeqG1V9Rx96bE6EaDhkxIUHeziqxNPVoNFwYvDlFZkxknU1nKNMGEkxK1motVXR3AOrQ7rV00ULDgUYEyAXlezwPI3Pm2LCZ+da4HJvYFJxObBU9sKCHHHByYwPh06rkWwqu5IZ+H89KzkSx2racbq+C4/dMDNIoxoZMwUYS4qkKAOijCHo7HPgXGM35qaagj2kQdgcLlS6K7uTMCYAEjiyw/M0Xt/RC0Aoof7Dq6eJzwcbz/gO1sdmekKEpFPZ1QQLGD3dIF13Q38NHDqJSwGO4zArOQqfV7biTEOXJAVOZUsPHC4eEYYQpJiMwR4OIQFI4MgMz9N4cpQR0WE6tFvsONfYHXTLDcOz5sq8KcJC2GaxSTqVXU3MShHcDacbOoM8kqFxunixQjedxKXDzORIUeBIEZZBRZWvCQbF4MgY4VQlnHBP1Utrs2KppF9d6AAAHKpoJXEjEdicqW3rRVefPcijESjyiCG70NYLq8MFfYgGabFhKC41o8htESSCh9QDjZnVbwa5NQk3JHBkzmz3afxUvfQWnUeuny7el3oqu5qIDtMjOUow4Z+VyGbF3JrFpWax1H52QgRe2n0Oz7ljuojgMmtAqrjU6G/RQG5NQoAEjsyZnSxdd8P//fcp8b6UU9nVyEzR8ieNzYpZ/J4rOYst+ysBAC6eJ7emhJjhnjP1HX3osEjP8tffZFOw4JDljyCBI3P6LTid4PngBxgzikvN2LyvAgCQOyVK0uXd1cisFBZoLB1hzETOp+ZmAIKlgMSNdIgy6sRmuVJxUzHLX1HJWVSwDKqkSDGhgSx/6oYEjszJSYqAhgPaLHY0dlmDPRwA/ansC7NiAQBzUqK8TugkcoIPs/xJzd1QWJADtiWRW1N6sPiWMxIRxmxdeb7UDIeLR7hei38eriHLHwGABI7sMeq0mJYgLDpfSyTQmKWysxRxllIq5R42aoO5qE7Xd0nK8vfcR2dkUaFbrcwUXeLSEcaFBTm45ZIUAIDF5pR8bz4icFCauAKYlRyJc43dOF3fhetmJgZ7OGIq+6LflgIA5qZGic/RoiMNshMiEKLh0GV14EJ7L6bGhAV7SCguNaP443MAgLhwPe7Nz5BM8UpCgAUaSyU4nTEtXqiSzgPQazU0XwgAZMFRBJ5xOFKhpduKhs4+cFx/3RVCOuhDNGJvMCm4qZhbs2C2INDnpEbh0aUzyK0pMWZ6FImUkuXvX8frAAgxOTani+YLAYAsOIpgtgQDRk/WCWPJjAtHhIGmmRSZmRyJ0w1dON3QhYLZSUEdC3Nr1nf0AQByp/S7NdnzRPARLX99DtR39CE1Ovid3p/fdRYVLRYAwN9/lI/PzjWT5Y8AQAJHETALzvmmHvTZnTDqtEEeUb/AmZNK1hupMis5Cu+jThKWP+bW/OaL+wAAuR6tAGiTkg76EA2mJYTj7MVunGnoCrrAKS41o2iXYK3RcEJCQ15GDACQyCHIRaUEkqOMMIXq4HTxOOduUhhsWMDzXBI4kkVqhdvsThdOuceSO4XmjVSRUqCx08Xjm5elAhAK/IXqhcMdJTQQAAkcRcBxnOimksJpHABO1gktGqTYlI8QYLVwypsFy1+wOdfYDZvDhUhjCNJjgx/0THjDiur1C+P+tSZYRfXWLZuBjDghwJi5NRmFBTlevfsI9UECRyHMktCpqsfqEItuzaEAY8kiNcvfCXffsjkpUdQsUYKwonosg+qMu3JwsIvqsXkzj6x+xAAoBkchzJFQJtXphk7wPJAYaUBCpCHYwyGGoMi9Ic1MjsTnFUKHaHYCLi41w+niA376ZXFbA0/ihDRgsSwstuV8YzeKSs7i+dLg1p0RBc5UmjeEN2TBUQizPFxUwU7f/LqO4m+kDjuN2xyCa4pl4AXzNM7cmhR/I10KC3KwbqkgZGxOV9DFTWNnHxq7rNBw/ckWBMEggaMQZiRFSqZlw0lR4NCJSqqwIMxjNYKoON3QJYqbYGxYLhffb8GheSNpHl06Q2ynoQ1yO42v3Nab7IQIhOnJIUF441eB09bWhlWrVsFkMsFkMmHVqlVob28f8TUcxw15+5//+R/xmmuvvXbQ83fffbc/P4qkKSo5i9c+KUeWu5qnp5sqGMF/lCIuDwoLcnD3gjQAwKfm5qD276lo6YHF5oRRpxFbjxDSpLjULLbTcAa5ncZXYvwNiWJiMH4VOPfccw+OHTuGnTt3YufOnTh27BhWrVo14mvq6+u9bq+//jo4jsPtt9/udd2aNWu8rnv11Vf9+VEkDXM3MLfCqXohCDAY7ga70yV2GiYXlfT59c1zxPs6bfBO40wUz06Jog7QEoatKbe6ez8lRBiCWmmaxd9Q3BYxFH6z6Z06dQo7d+7EwYMHsXDhQgDAn/70J+Tn5+PMmTOYOXPmkK9LTk72+vn999/Hddddh2nTpnk9HhYWNuhatTIw+O90Q2fQ3A3nm9ypvoYQpEmgvxExMpv3VYj37U7hNB4MkXOSbVTknpIsnmvKnVdMxb++rEerxYZHrp8etKJ6X1GAMTECfrPgHDhwACaTSRQ3ALBo0SKYTCbs379/TO9x8eJF/Pvf/8bq1asHPffWW28hPj4ec+fOxWOPPYauruCnRweTwoIcfGu+UPDq/WN1QXM3nLzgPomnRkFDJ3FJwzYstjnkZUQH7TR+ggKMJQ9rp1FYkIMUUyiSogxwungsyUkISlG9xq4+XOy0guOoHAUxNH6z4DQ0NCAxcXBn68TERDQ0NIzpPf785z8jMjIS3/nOd7we/973voesrCwkJyfjxIkT2LBhA44fP46SkpIh38dqtcJq7Q+87ewMfiq1P/j/bpmL98qEpnPBcjeI8Te04Egaz9N4anQoHvvHcQCc2NwSCNxpnOd5CkyXAQPLBlyWFo0PT17EsZq2IFn9hDmTnRCBcOp3RwzBuC04Tz755LCBwOx2+PBhABiyWBfP82Mu4vX666/je9/7HoxGo9fja9aswdKlS5Gbm4u7774b//znP7Fr1y4cPXp0yPfZuHGjGOhsMpmQlpY2zk8tD948WCXeZ+6GQFDkcer/up5VMBYETrAqnBIj43kavzw9GoBg7n/wmuyAn8YvtPei3WKHTsthRlJkwH4vMTkuSxN6PpVVtwfl91OAMTEa45a9Dz/88KgZS5mZmfjyyy9x8eLFQc81NTUhKWn0zsWffvopzpw5g3feeWfUay+//HLodDqYzWZcfvnlg57fsGED1q9fL/7c2dmpOJHDTuRzUqPwdV0nFmTFBOwkzoKceZ73qIFj8rISENLC8zSeFR+O6DAd2i12fF3fGfDT+An3SXxGUiT0IVS5Qi7MdwvjYzXtQfn9X1GAMTEK4xY48fHxiI+PH/W6/Px8dHR04PPPP8eCBQsAAIcOHUJHRwcWL1486us3b96MvLw8XHrppaNee/LkSdjtdqSkpAz5vMFggMGg3Iq6nkIiIdKADe9+BQ0XOHfDwCBnvVaDD082BL0IGDE2OI7D/LRo7D7ThKNVbbgsLdrvv5NVUi4syOkv8Jca3ErKxPiYN8UEDQfUd/ThYmcfkqKMo7/Ih4gZVJStSQyD345Ls2fPxo033og1a9bg4MGDOHjwINasWYNbbrnFK4Nq1qxZ2L59u9drOzs78Y9//AMPPPDAoPc9f/48nn76aRw+fBiVlZXYsWMH7rzzTsyfPx9XXXWVvz6OpPF0N1yRIZiNj9d04MfXBs7dUFiQI6aO2iVQ4ZQYH5enu90NATqNM6tfcanZo0VDVND7GhFjJ9wQIroUA+2mau62or6jDxwHzCULDjEMfo3Meuutt1BYWIjly5cDAG677Ta8+OKLXtecOXMGHR0dXo9t3boVPM/ju9/97qD31Ov1KC0txfPPP4/u7m6kpaXh5ptvxhNPPAGtVuu/DyNhPE+62QkRiDKGoLPPgVMBdjewExwPwYpD4kY+XO4Wxker2gLy+zytfuF64e/2ZF0ntn5RQ8JYRsxPj8bphi6U1bThxlz/lu3wtPox91RWfDgiDCFk9SOGxK8CJzY2Fm+++eaI1wzVN+mHP/whfvjDHw55fVpaGvbu3euT8SkRjYbD5Rkx2HOmCUeq2nDJ1OiA/e4PjgsZXFoNB5vTFbSaKsT4uWSqCRwnBPw2dvYhMQDuhsKCHPRYHXj1k3IAIHEjQ+anxeDtz2twLAAWHGb1AyC2ipg3hWL9iOGhiD4FwtxURwJ0GgeA//nwtNgD67OfXy/G/wSzjDsxdiKNOsx0uxuOVgdu3njG+5DVT35c5pGB529XOOuf9lzJWbx/7AIAoN1iC2qLEULakMBRIJcHWOAUl5rx0u7zAICMuDAkm4xeixGJHHkwPz3wab+vfiLMGw0H0epHyIfshAhEGEJgsTlx9qL/i62ydeVcUw8AYO/ZZhI3xLCQwFEgl06NhlbDob6jD3XtvX7/fU4Xjyszhc1xQWas+DhbjAJd4ZSYGKweTqAsOMWlZrGb+cvfyyNBLEO0Gg6XuCthB0oY35E3VbwfzP5phPQhgaNAwg0hmJ0iuBsCYcVZ5yFiFk6L83qusCCHAv9kArPgfFnbAZvD5dffxeImAIDjgIVZsWT1kyn99XACI4yf/OAkACEOJ5AFTQn5QQJHoVyRIVhSAiFwLDYHvqwVTuILs2JHuZqQKtPiw2EK1cHqcOF0g3/bmThdPG6eJ5QVmJUchZhwPQCy+skRVtE4EAX/ikvN+OhroYDsQ9dlkyAmRoQEjkIJZBzO0ap2OFw8Uk1GTI0J9fvvI/yDRsOJp3F/p4uvWzYD0WE6AMCiad6imKx+8qGo5CwOV7YCAMyN3ejqs4vP+bpNC7P6hRuEsgKLs+PJ6keMCAkchZLnFjhf13fCYnP49Xd9XtECQHBPjbXPGCFNWMG/owGIpzhQLsyb/AFuTUI+aDUcXv2kHJHGEPA8REuuPwo2Ol087l+ciR6rE/oQjbjGkdWPGA5qwapQpkSHIsVkRH1HH47XdCA/23+byMEK4QRH7il5U1RyFrVtFgBA2YB4Cl8XUmvs7EN5U487/oYEjlwZ2KblWE07jlS1+SV1e92yGfjLgUoAQikMo66/sCsFGhNDQRYcBdPvpmr12+/osztF3/sCEjiyRqvhsO2oUF+kprUXTe66Rv44jTPrzZyUKJjcripCnhQW5OCaGUJ/wt9/dMavdWk+O9cMALhq+uj9EAmCLDgKhJU0z0uPwb+/rPeKw/H1Sfx4TTtsDhcSIg3Iig/3yXsSwWHgafxodRvONHT5ZcM6SO4pRfHY8lnYe3YfeN5/qdtOF48D54V5s9iPFmlCOZAFR4GwkubnmroBCPEULhfvl5P4IQ/3FMXfyJ/CghzMmyJ0Z/7xm0f8dho/WC7MG3+6TonA8fHpi+J9f6Vun6zrQGefA5GGEMyjBpvEGCCBo0BY0N3fDlUjRMOho9eOp/510i+b1ecUf6M4fnh1NgDAxfunfUJDRx8qmnug4YArad7InuJSM4p2mZHrFsbz06L9ktX02bn+ZIYQLW1dxOjQLFEoTOQ43JkFfz5Q5XNxY3e6RPfXwAJ/hHzxrIHjj/YJB8qFOIrcKSZEGSn+Rs54NrosvF5YW5p7rFi3NMfnImf/eRZ/Q2sNMTYoBkfBFBbk4Hl3zA0H32QasPiewoIcfFnbgV67E7HheuQkRvg8vocIPKyvWEZsGKpaLViSEy/G5PhKHLM4ikUkimWP08WLB6ceqwP6EA1qWntx8yUp4DjOZ6nbVocTX7jr7SzOpgBjYmyQwFEwTHAAAA/gmf+cxs9XzJrUe7L4HgDQuc3EV2bG4IWPz4knOUKeeJ7GY8L1+PV7J9BjdYiF1ICJiRxPUQx4xN9MiyNRLHM8v7dwQwjyp8Vh79kmlJ5q9Km1+GhVO/rsLsRHGDAjKcJn70soG3JRKRTPzWpOiuAb/+Pe85M2GXtWDn3ni2oAgNXu8mtqKBEYPE/jy2YnAQDKatrx3QXpkyqkxkRxcakZF9p7Ud1qgVbD4XBVq8+D3ongUjA7EQBQerrRp+97wO2eWpxNxUSJsUMWHAXi5RcvyAEHoaJxVlyYT9wNhQU5sDtdeOHjcwCAPWebSNwoAM/TeLLJiEummvBlbQdKT12c9HwBhPTzk3VCfE9ChAEv7T5P80ZhXDczEcBJHKlqQ7vFhugw/YTeZ6DV7zO3W/Oq6WT1I8YOWXAUiOdJHABWzEsGANS29+In12X7xC8+PbHfTOyPTBsi+DArTsnXF0e5cnSY5e/Dkw0AgIbOPhI3CiQtNgwzkyLhdPHYe7Zpwu/jafXrtjpw3F1M9OzFbrL6EWOGBI4CWTdg45ieGInshHDYnTxmJEX65OTDmuhpOP9k2hDBZ9lcQeB8eq4ZPdbJ9zP74dXTxPshGv8UgyOCz/VuN9XHk3BTebrCf/XeV3C4eJhCddi8r4KEMTFmSOCohBW5KQCA/3zVMOn32rjjFCpbhJ5Fu9ZfQ918FcrMpEikxYbC5nDhU/PET+OM9X8/Lt53uPxTDI4IPgWzBIGz50wTHE7XhN+HiZz3yuoAAB29dhI3xLgggaMSbswV3FR7zjZOqrt4cakZr35SDgC4NC0a0xIivE5btGkpB47jsGy2MG8+mqSbqrjUjB1f1QMAzRcFU1RyFvvONSM6TIeOXrtXV/riUrNo+R0rq7+RJd4nqx8xXkjgqIS5qVGYGhOKPrsLe89M/DTudPFIjDQAAL4zf4r4ONu0fFX3ggg+RSVn0WW1AxDcDZ6n8fFsVizonXFn3lQSOQpFq+GwaZcZKSYjAKDU3cJhom1i1m4tE++T1Y8YLyRwVALHcVjhtuLsPDlxN9Wtl6agscuKEA2HWy9N9XqusCCHMhsUhFbD4R+Ha2HUadBuseOwu2r1eDcrp4sXm2ouzo5DWmwYABLFSoR9p6fquwAAH59qHJTVOVaKS80oOSXE8fziplkkiIlxQ2niKqGo5Cy6+gTX1MenGmF1OGEI0QIYX4fxd49eAABcOzMBseETSwEl5MHA7uIlX1/E5xWt496sHi3IwdX/sxsAsPKKtCF/B6EcCgty0Gd34uU952Fu7J6wuGHzTq/V4I68NHG98XVlbUK5kAVHJWg1HLZ+UYNwgxZdVgc+OycUzhrPadzl4vH+MSHg79vzp/p1vIQ0KCzIwa2XCAHqm/dVTGizOljegtq2XkQaQ8RYMELZ/OzGWWArioYbvxhxunjkpgoFSlfMSxbFDVn9iPFAAkclsIWhx+oEAOw80TAm03GRh0n4UEUrLrQLG1XB7MQJBQ0S8uOZOy4R708k0PPvh2sAALddmgqjTuvTsRHSpLjUDCZBXDzw2x2nxvX61UuycL6pBwDwvYUZXs+RK5wYKyRwVERhQQ7uyBMsL38/XDum07hnwa3tZbUAgJvnpeC1T8qp4JZK+N9PK8T7DhePTbtGFrWeorij147/nBBivlZekUaiWAWwg9O6pTm4IiMGAPDaJ+Xjip15r+wCeu1O5CRG4MrMGH8NlVA4FIOjMp65/RJsO1ILHmMzHXvGYejdzTVdPE+9p1QC26x+fE023jxUha4+BzbtMkPDDW/J8WzIGhuuh9XhwsykSOw504iiXWZqyKpgBlqFr8yMxT3/ewhajhsxdsazNQPP83jroNDn7nsL0/HCx+eoNQMxIciCozJe2n3Oy3T8+LYvR31NYUEObpybDJs7Tfjvh2tJ3KgAz83q5ytm4eHrpgMAIgwhI2azeKaAv/ixcE2yySCKG5o3ymVgm5j87DgszIqFk+dxyVTTsLEznpbiI1VtOHOxC0adBo1dVrIUExOGLDgqwnPDqm614J9HarH1ixokm4xYu3T401FNqwUHylvEn6n3lDoYuFndtzgTfzlQhQvtvfjG9LgRAz0LC3Jwob0X73whxN/sPdtM4kYFDLSycByHqTGhOFQBnKrvxMvfu9zr+YEZnM+VnMV/TkQCALITIvDyHmrISkwcsuCohIGm45gwHfQhwte/aZfZ6zTuGSdhdTjxk78dRUevUPBNp+Wo95RKGNjTzKjTIneKkNlyvKYD9+Z7B396zpuaVgt2eVQ/JlGsXjLiwgEAdiePl3afEx8fmMFZWJCDuxekiTV0TtZ1krghJoVfBc5vfvMbLF68GGFhYYiOjh7Ta3iex5NPPonU1FSEhobi2muvxcmTJ72usVqteOSRRxAfH4/w8HDcdtttqK2t9cMnUA4DT+ORRh1sDsHlZAjRoNvqQFHJWdzzp4Nei85v/n0KX9Z2AAAuT4+G+Tc3UcEtFTMnRRA4XVYHXvj4nBhQ7LlZdVjsuP+Nz9HSYwNAoljtFBbk4K4rhfpH73xRgyc/OCmuM55r0rGadrx75IL4OhLFxGTxq8Cx2Wy488478eMf/3jMr3n22Wfx3HPP4cUXX8QXX3yB5ORkLFu2DF1dXeI1a9euxfbt27F161bs27cP3d3duOWWW+B0Ov3xMRTBwNN4YUEO1rp/tjpc6Oy144vKVuw/34LF2XEoLMjBB8fr8JcDVeJrrp2ZKL6WRI46eXTpDDET7y8HKtHZa8dzJWfFzerBa7Lx4JtHxBTfKzJiSBQTeOb2S5ARFwYXD2zZX+m1zgDAl7XtWPnqATHOj0Qx4Qs4nuf9XjFpy5YtWLt2Ldrb20e8jud5pKamYu3atfj5z38OQLDWJCUl4ZlnnsGPfvQjdHR0ICEhAX/9619x1113AQDq6uqQlpaGHTt24IYbbhh1PJ2dnTCZTOjo6EBUVNSkP5+c+fm2L8U4CQDIjAtDZYsF312QjvePXYDFJojGoUzF46mATCiLa/9nt9hRnvHwddPR0NmHfx4RrKmXTDXhg4e/IT4/0ZL9hDI4VtOOb730mddjq6/Kwrcvn4Lb/7gfVrdF+eHrpuOxG2bSfCGGZDz7t6RicCoqKtDQ0IDly5eLjxkMBlxzzTXYv38/AODIkSOw2+1e16SmpiI3N1e8hhg7z9x+CTwTFNim9fbn1aK4Wbs0Z8gFhgpuqZc//2ABBua1vLj7nChuZqdEeokbgKrQqp1PzgpNfj3nzebPKnDLC/sGiRuALMXE5JFUFlVDg1AQLCkpyevxpKQkVFVVidfo9XrExMQMuoa9fiBWqxVWq1X8ubOz05fDljXFpWa4eKFCrcPFY1p8OKwOFy609wIQTMUjZVgR6uT9Y3XgIcwPu5NHdJgO7RYhEF2r4fCfR68e8nV0ElcnA60xj2/7Els9LMeAt7hhsPlCopiYCOO24Dz55JPgOG7E2+HDhyc1KI7zPhvyPD/osYGMdM3GjRthMpnEW1pa2pDXqQ3PRefcb4U4ifLmHmTECd2e2eZFpyfCE895w+JrmLgJ0XBwumjOEP0M5Wr63e2XYHG20GGeLdssq3MgZCkmJsq4LTgPP/ww7r777hGvyczMnNBgkpOFRnwNDQ1ISUkRH29sbBStOsnJybDZbGhra/Oy4jQ2NmLx4sVDvu+GDRuwfv168efOzk7Vi5yhFp3CghwcLG8RAwD/tmaRV1dfOn0To8VFDOxATnOGGJjBCQjziK0zV2bGelW/pjlD+IpxC5z4+HjEx8f7YyzIyspCcnIySkpKMH/+fABCJtbevXvxzDPPAADy8vKg0+lQUlKClStXAgDq6+tx4sQJPPvss0O+r8FggMFg8MuY5cpYFh2ANizCm4HzxlPwsOc9i7YBNGfUzkDry0gimeYM4Uv8GoNTXV2N1tZWVFdXw+l04tixYwCA6dOnIyIiAgAwa9YsbNy4Ed/+9rfBcRzWrl2L3/72t8jJyUFOTg5++9vfIiwsDPfccw8AwGQyYfXq1fjpT3+KuLg4xMbG4rHHHsO8efOwdOlSf34cRTGUyXco0QOQH5zoZ+C8oTlDjBeaM0Sg8Gua+P33348///nPgx7fvXs3rr32WmEAHIc33ngD999/PwAhluapp57Cq6++ira2NixcuBAvvfQScnNzxdf39fXhv//7v/G3v/0Nvb29KCgowMsvvzxmtxOliRMEQRCE/BjP/h2QOjhSgwQOQRAEQcgP2dbBIQiCIAiC8AUkcAiCIAiCUBwkcAiCIAiCUBwkcAiCIAiCUBwkcAiCIAiCUBwkcAiCIAiCUBwkcAiCIAiCUBwkcAiCIAiCUBwkcAiCIAiCUBx+7UUlVVjx5s7OziCPhCAIgiCIscL27bE0YVClwOnq6gKAMfeuIgiCIAhCOnR1dcFkMo14jSp7UblcLtTV1SEyMhIcx/n0vTs7O5GWloaamhpF9rlS+ucDlP8Z6fPJH6V/Rvp88sdfn5HneXR1dSE1NRUazchRNqq04Gg0GkydOtWvvyMqKkqxExdQ/ucDlP8Z6fPJH6V/Rvp88scfn3E0yw2DgowJgiAIglAcJHAIgiAIglAcJHB8jMFgwBNPPAGDwRDsofgFpX8+QPmfkT6f/FH6Z6TPJ3+k8BlVGWRMEARBEISyIQsOQRAEQRCKgwQOQRAEQRCKgwQOQRAEQRCKgwQOQRAEQRCKgwTOOPnNb36DxYsXIywsDNHR0UNeU11djVtvvRXh4eGIj49HYWEhbDbbiO9rtVrxyCOPID4+HuHh4bjttttQW1vrh08wPvbs2QOO44a8ffHFF8O+7v777x90/aJFiwI48rGTmZk5aKyPP/74iK/heR5PPvkkUlNTERoaimuvvRYnT54M0IjHR2VlJVavXo2srCyEhoYiOzsbTzzxxKhzUsrf4csvv4ysrCwYjUbk5eXh008/HfH6vXv3Ii8vD0ajEdOmTcMrr7wSoJGOn40bN+LKK69EZGQkEhMT8a1vfQtnzpwZ8TXD/Z2ePn06QKMeO08++eSgcSYnJ4/4Gjl9f8DQawrHcfjJT34y5PVS//4++eQT3HrrrUhNTQXHcXjvvfe8np/oerht2zbMmTMHBoMBc+bMwfbt2306bhI448Rms+HOO+/Ej3/84yGfdzqduPnmm9HT04N9+/Zh69at2LZtG37605+O+L5r167F9u3bsXXrVuzbtw/d3d245ZZb4HQ6/fExxszixYtRX1/vdXvggQeQmZmJK664YsTX3njjjV6v27FjR4BGPX6efvppr7H+6le/GvH6Z599Fs899xxefPFFfPHFF0hOTsayZcvEPmdS4vTp03C5XHj11Vdx8uRJFBUV4ZVXXsEvfvGLUV8rxe/wnXfewdq1a/HLX/4SZWVlWLJkCVasWIHq6uohr6+oqMBNN92EJUuWoKysDL/4xS9QWFiIbdu2BXjkY2Pv3r34yU9+goMHD6KkpAQOhwPLly9HT0/PqK89c+aM1/eVk5MTgBGPn7lz53qN86uvvhr2Wrl9fwDwxRdfeH2+kpISAMCdd9454uuk+v319PTg0ksvxYsvvjjk8xNZDw8cOIC77roLq1atwvHjx7Fq1SqsXLkShw4d8t3AeWJCvPHGG7zJZBr0+I4dO3iNRsNfuHBBfOztt9/mDQYD39HRMeR7tbe38zqdjt+6dav42IULF3iNRsPv3LnT52OfDDabjU9MTOSffvrpEa+77777+G9+85uBGdQkycjI4IuKisZ8vcvl4pOTk/nf/e534mN9fX28yWTiX3nlFT+M0Pc8++yzfFZW1ojXSPU7XLBgAf/ggw96PTZr1iz+8ccfH/L6n/3sZ/ysWbO8HvvRj37EL1q0yG9j9CWNjY08AH7v3r3DXrN7924eAN/W1ha4gU2QJ554gr/00kvHfL3cvz+e5/lHH32Uz87O5l0u15DPy+n7A8Bv375d/Hmi6+HKlSv5G2+80euxG264gb/77rt9Nlay4PiYAwcOIDc3F6mpqeJjN9xwA6xWK44cOTLka44cOQK73Y7ly5eLj6WmpiI3Nxf79+/3+5jHwwcffIDm5mbcf//9o167Z88eJCYmYsaMGVizZg0aGxv9P8AJ8swzzyAuLg6XXXYZfvOb34zovqmoqEBDQ4PX92UwGHDNNddI7vsajo6ODsTGxo56ndS+Q5vNhiNHjnj93wPA8uXLh/2/P3DgwKDrb7jhBhw+fBh2u91vY/UVHR0dADCm72v+/PlISUlBQUEBdu/e7e+hTRiz2YzU1FRkZWXh7rvvRnl5+bDXyv37s9lsePPNN/GDH/xg1ObOcvn+PJnoejjc9+rLNZQEjo9paGhAUlKS12MxMTHQ6/VoaGgY9jV6vR4xMTFejyclJQ37mmCxefNm3HDDDUhLSxvxuhUrVuCtt97Cxx9/jD/84Q/44osvcP3118NqtQZopGPn0UcfxdatW7F79248/PDD2LRpEx566KFhr2ffycDvWYrf11CcP38eL7zwAh588MERr5Pid9jc3Ayn0zmu//uh/iaTkpLgcDjQ3Nzst7H6Ap7nsX79enzjG99Abm7usNelpKTgtddew7Zt2/Duu+9i5syZKCgowCeffBLA0Y6NhQsX4i9/+Qs+/PBD/OlPf0JDQwMWL16MlpaWIa+X8/cHAO+99x7a29tHPBTK6fsbyETXw+G+V1+uoarsJj6QJ598Ek899dSI13zxxRejxpwwhlLpPM+Pqt598ZqxMpHPXFtbiw8//BB///vfR33/u+66S7yfm5uLK664AhkZGfj3v/+N73znOxMf+BgZz+dbt26d+Ngll1yCmJgY3HHHHaJVZzgGfjf+/L6GYiLfYV1dHW688UbceeedeOCBB0Z8bbC/w5EY7//9UNcP9bjUePjhh/Hll19i3759I143c+ZMzJw5U/w5Pz8fNTU1+P3vf4+rr77a38McFytWrBDvz5s3D/n5+cjOzsaf//xnrF+/fsjXyPX7A4RD4YoVK7ys+gOR0/c3HBNZD/29hpLAgbCI3H333SNek5mZOab3Sk5OHhQk1dbWBrvdPkiter7GZrOhra3Ny4rT2NiIxYsXj+n3jpeJfOY33ngDcXFxuO2228b9+1JSUpCRkQGz2Tzu106EyXynLFPo3LlzQwoclvHR0NCAlJQU8fHGxsZhv2N/MN7PWFdXh+uuuw75+fl47bXXxv37Av0dDkV8fDy0Wu2gU95I//fJyclDXh8SEjKigA02jzzyCD744AN88sknmDp16rhfv2jRIrz55pt+GJlvCQ8Px7x584adV3L9/gCgqqoKu3btwrvvvjvu18rl+5voejjc9+rLNZQEDoRFMz4+3ifvlZ+fj9/85jeor68Xv+yPPvoIBoMBeXl5Q74mLy8POp0OJSUlWLlyJQCgvr4eJ06cwLPPPuuTcQ1kvJ+Z53m88cYbuPfee6HT6cb9+1paWlBTU+P1B+BPJvOdlpWVAcCwY83KykJycjJKSkowf/58AIKffe/evXjmmWcmNuAJMJ7PeOHCBVx33XXIy8vDG2+8AY1m/N7pQH+HQ6HX65GXl4eSkhJ8+9vfFh8vKSnBN7/5zSFfk5+fj3/9619ej3300Ue44oorJjSX/Q3P83jkkUewfft27NmzB1lZWRN6n7KysqB+V2PFarXi1KlTWLJkyZDPy+378+SNN95AYmIibr755nG/Vi7f30TXw/z8fJSUlHhZ0D/66CPfHup9Fq6sEqqqqviysjL+qaee4iMiIviysjK+rKyM7+rq4nme5x0OB5+bm8sXFBTwR48e5Xft2sVPnTqVf/jhh8X3qK2t5WfOnMkfOnRIfOzBBx/kp06dyu/atYs/evQof/311/OXXnop73A4Av4Zh2LXrl08AP7rr78e8vmZM2fy7777Ls/zPN/V1cX/9Kc/5ffv389XVFTwu3fv5vPz8/kpU6bwnZ2dgRz2qOzfv59/7rnn+LKyMr68vJx/5513+NTUVP62227zus7z8/E8z//ud7/jTSYT/+677/JfffUV/93vfpdPSUmR3OfjeSEjb/r06fz111/P19bW8vX19eLNE7l8h1u3buV1Oh2/efNm/uuvv+bXrl3Lh4eH85WVlTzP8/zjjz/Or1q1Sry+vLycDwsL49etW8d//fXX/ObNm3mdTsf/85//DNZHGJEf//jHvMlk4vfs2eP1XVksFvGagZ+xqKiI3759O3/27Fn+xIkT/OOPP84D4Ldt2xaMjzAiP/3pT/k9e/bw5eXl/MGDB/lbbrmFj4yMVMz3x3A6nXx6ejr/85//fNBzcvv+urq6xL0OgLhmVlVV8Tw/tvVw1apVXpmOn332Ga/Vavnf/e53/KlTp/jf/e53fEhICH/w4EGfjZsEzji57777eACDbrt37xavqaqq4m+++WY+NDSUj42N5R9++GG+r69PfL6iomLQa3p7e/mHH36Yj42N5UNDQ/lbbrmFr66uDuAnG5nvfve7/OLFi4d9HgD/xhtv8DzP8xaLhV++fDmfkJDA63Q6Pj09nb/vvvsk9XkYR44c4RcuXMibTCbeaDTyM2fO5J944gm+p6fH6zrPz8fzQmrkE088wScnJ/MGg4G/+uqr+a+++irAox8bb7zxxpBzduD5Rk7f4UsvvcRnZGTwer2ev/zyy71SqO+77z7+mmuu8bp+z549/Pz583m9Xs9nZmbyf/zjHwM84rEz3HflOf8GfsZnnnmGz87O5o1GIx8TE8N/4xvf4P/9738HfvBj4K677uJTUlJ4nU7Hp6am8t/5znf4kydPis/L/ftjfPjhhzwA/syZM4Oek9v3x9LYB97uu+8+nufHth5ec8014vWMf/zjH/zMmTN5nU7Hz5o1y+eCjuN5d7QWQRAEQRCEQqA0cYIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFAcJHIIgCIIgFMf/D6TbNr/nUq9MAAAAAElFTkSuQmCC\n",
|
||
"text/plain": [
|
||
"<Figure size 640x480 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"filenames": {
|
||
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/linalg_44_1.png"
|
||
}
|
||
},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"%matplotlib inline\n",
|
||
"\n",
|
||
"import numpy as np\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"from scipy import sparse\n",
|
||
"eye = np.eye(4)\n",
|
||
"print(eye)\n",
|
||
"sparse_mtx = sparse.csr_matrix(eye)\n",
|
||
"print(sparse_mtx)\n",
|
||
"x = np.linspace(-10,10,100)\n",
|
||
"y = np.sin(x)\n",
|
||
"plt.plot(x,y,marker='x')\n",
|
||
"plt.show()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ed00bf63",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Other Matrix and Vector Operations\n",
|
||
"\n",
|
||
"The following examples show how to compute various quantities like the **mean** value of a matrix or a vector and how to use functions like **reshape** and **ravel**. These are all useful quantities when scaling the data and preparing the data for various machine learning algorithms and when calculating quantities like the mean squared error or the variance."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 16,
|
||
"id": "a12e8986",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"The test matrix:[[ 1. 2. 3.]\n",
|
||
" [ 4. 5. 6.]\n",
|
||
" [ 7. 8. 9.]\n",
|
||
" [10. 11. 12.]]\n",
|
||
"This is the total mean summed over all elements:6.5\n",
|
||
"This is the mean for each column:[[5.5 6.5 7.5]]\n",
|
||
"This is the mean value for each row:[[ 2.]\n",
|
||
" [ 5.]\n",
|
||
" [ 8.]\n",
|
||
" [11.]]\n",
|
||
"This is the mean value for each row with keepdims false:[ 2. 5. 8. 11.]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"\"\"\"\n",
|
||
"Simple code that tests various numpy functions\n",
|
||
"\"\"\"\n",
|
||
"\n",
|
||
"import numpy as np\n",
|
||
"# Simple test-matrix of dim 3 x 4\n",
|
||
"a = np.array([ [1, 2, 3], [4, 5, 6], [7, 8, 9],[10, 11, 12]],dtype=np.float64)\n",
|
||
"print(f\"The test matrix:{a}\")\n",
|
||
"# This is the total mean summed over all elements, which here has to be 6.5\n",
|
||
"print(f\"This is the total mean summed over all elements:{np.mean(a,dtype=np.float64)}\")\n",
|
||
"# This is the mean for each column, it returns an array with the mean values for each column. It returns a row-like vector\n",
|
||
"print(f\"This is the mean for each column:{np.mean(a, axis=0, keepdims=True,dtype=np.float64)}\")\n",
|
||
"# This is the mean value for each row, it returns an array via the keepdims option which is a column-like vector if\n",
|
||
"# keepdims=True. Else it return a row-like vector\n",
|
||
"# Try setting keepdims=False\n",
|
||
"print(f\"This is the mean value for each row:{np.mean(a, axis=1, keepdims=True,dtype=np.float64)}\")\n",
|
||
"# We print then the mean value for each row by setting keepdims=False\n",
|
||
"print(f\"This is the mean value for each row with keepdims false:{np.mean(a, axis=1, keepdims=False,dtype=np.float64)}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "596d78a6",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Another useful function is the **ravel** function, which returns a flattened array as shown in the example here."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"id": "fe01c225",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Flatten the matrix:[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]\n",
|
||
"Reshape the matrix to a one-dim array:[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]\n",
|
||
"[ 1. 4. 7. 10. 2. 5. 8. 11. 3. 6. 9. 12.]\n",
|
||
"[ 1. 4. 7. 10. 2. 5. 8. 11. 3. 6. 9. 12.]\n",
|
||
"[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Ravel return a contiguous flattened array.\n",
|
||
"print(f\"Flatten the matrix:{np.ravel(a)}\")\n",
|
||
"# It is the same as reshaping the matrix into a one-dimensional array\n",
|
||
"print(f\"Reshape the matrix to a one-dim array:{a.reshape(-1)}\")\n",
|
||
"# ‘C’ means to index the elements in row-major, C-style order, with the last axis index changing fastest, back to the first axis index changing slowest.\n",
|
||
"# ‘F’ means to index the elements in column-major, Fortran-style order, with the first index changing fastest, and the last index changing slowest \n",
|
||
"print(np.ravel(a, order='F'))\n",
|
||
"# When order is ‘A’, it will preserve the array’s ‘C’ or ‘F’ ordering\n",
|
||
"# ‘A’ means to read the elements in Fortran-like index order if a is Fortran contiguous in memory, C-like order otherwise.\n",
|
||
"# ‘K’ means to read the elements in the order they occur in memory, except for reversing the data when strides are negative. By default, ‘C’ index order is used.\n",
|
||
"# Transposing it\n",
|
||
"print(np.ravel(a.T))\n",
|
||
"print(np.ravel(a.T, order='A'))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "545df59a",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Gaussian Elimination\n",
|
||
"\n",
|
||
"We start with the linear set of equations"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "796d7554",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}\\mathbf{x} = \\mathbf{w}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "35a9c235",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We assume also that the matrix $\\mathbf{A}$ is non-singular and that the\n",
|
||
"matrix elements along the diagonal satisfy $a_{ii} \\ne 0$. Simple $4\\times 4 $ example"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "432d4f8e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{bmatrix}\n",
|
||
" a_{11}& a_{12} &a_{13}& a_{14}\\\\\n",
|
||
" a_{21}& a_{22} &a_{23}& a_{24}\\\\\n",
|
||
" a_{31}& a_{32} &a_{33}& a_{34}\\\\\n",
|
||
" a_{41}& a_{42} &a_{43}& a_{44}\\\\\n",
|
||
" \\end{bmatrix} \\begin{bmatrix}\n",
|
||
" x_1\\\\\n",
|
||
" x_2\\\\\n",
|
||
" x_3 \\\\\n",
|
||
" x_4 \\\\\n",
|
||
" \\end{bmatrix}\n",
|
||
" =\\begin{bmatrix}\n",
|
||
" w_1\\\\\n",
|
||
" w_2\\\\\n",
|
||
" w_3 \\\\\n",
|
||
" w_4\\\\\n",
|
||
" \\end{bmatrix}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c06fa57b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4e8d9bd5",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{11}x_1 +a_{12}x_2 +a_{13}x_3 + a_{14}x_4=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "112c2488",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4=w_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4774bcd7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4=w_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5bf16627",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4da1a1c6",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The basic idea of Gaussian elimination is to use the first equation to eliminate the first unknown $x_1$\n",
|
||
"from the remaining $n-1$ equations. Then we use the new second equation to eliminate the second unknown\n",
|
||
"$x_2$ from the remaining $n-2$ equations. With $n-1$ such eliminations\n",
|
||
"we obtain a so-called upper triangular set of equations of the form"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "041684ac",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{11}x_1 +b_{12}x_2 +b_{13}x_3 + b_{14}x_4=y_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ba6214cb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{22}x_2 + b_{23}x_3 + b_{24}x_4=y_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2a1d0555",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{33}x_3 + b_{34}x_4=y_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "51570380",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"eq:gaussbacksub\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"b_{44}x_4=y_4. \\nonumber\n",
|
||
"\\label{eq:gaussbacksub} \\tag{1}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e51b3aa8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We can solve this system of equations recursively starting from $x_n$ (in our case $x_4$) and proceed with\n",
|
||
"what is called a backward substitution. \n",
|
||
"\n",
|
||
"This process can be expressed mathematically as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c78b600",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto1\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" x_m = \\frac{1}{b_{mm}}\\left(y_m-\\sum_{k=m+1}^nb_{mk}x_k\\right)\\quad m=n-1,n-2,\\dots,1.\n",
|
||
"\\label{_auto1} \\tag{2}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "1cd1015f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"To arrive at such an upper triangular system of equations, we start by eliminating\n",
|
||
"the unknown $x_1$ for $j=2,n$. We achieve this by multiplying the first equation by $a_{j1}/a_{11}$ and then subtract\n",
|
||
"the result from the $j$th equation. We assume obviously that $a_{11}\\ne 0$ and that\n",
|
||
"$\\mathbf{A}$ is not singular.\n",
|
||
"\n",
|
||
"Our actual $4\\times 4$ example reads after the first operation"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "34093ce7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{bmatrix}\n",
|
||
" a_{11}& a_{12} &a_{13}& a_{14}\\\\\n",
|
||
" 0& (a_{22}-\\frac{a_{21}a_{12}}{a_{11}}) &(a_{23}-\\frac{a_{21}a_{13}}{a_{11}}) & (a_{24}-\\frac{a_{21}a_{14}}{a_{11}})\\\\\n",
|
||
"0& (a_{32}-\\frac{a_{31}a_{12}}{a_{11}})& (a_{33}-\\frac{a_{31}a_{13}}{a_{11}})& (a_{34}-\\frac{a_{31}a_{14}}{a_{11}})\\\\\n",
|
||
"0&(a_{42}-\\frac{a_{41}a_{12}}{a_{11}}) &(a_{43}-\\frac{a_{41}a_{13}}{a_{11}}) & (a_{44}-\\frac{a_{41}a_{14}}{a_{11}}) \\\\\n",
|
||
" \\end{bmatrix} \\begin{bmatrix}\n",
|
||
" x_1\\\\\n",
|
||
" x_2\\\\\n",
|
||
" x_3 \\\\\n",
|
||
" x_4 \\\\\n",
|
||
" \\end{bmatrix} \n",
|
||
" =\\begin{bmatrix}\n",
|
||
" y_1\\\\\n",
|
||
" w_2^{(2)}\\\\\n",
|
||
" w_3^{(2)} \\\\\n",
|
||
" w_4^{(2)}\\\\\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f269239d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d4477133",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{11}x_1 +b_{12}x_2 +b_{13}x_3 + b_{14}x_4=y_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "25bdfc8c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a^{(2)}_{22}x_2 + a^{(2)}_{23}x_3 + a^{(2)}_{24}x_4=w^{(2)}_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "34b8408c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a^{(2)}_{32}x_2 + a^{(2)}_{33}x_3 + a^{(2)}_{34}x_4=w^{(2)}_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b8185f8d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a^{(2)}_{42}x_2 + a^{(2)}_{43}x_3 + a^{(2)}_{44}x_4=w^{(2)}_4, \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4a218e4e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto2\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation} \n",
|
||
"\\label{_auto2} \\tag{3}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ba424207",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The new coefficients are"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "10f17fb1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto3\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" b_{1k} = a_{1k}^{(1)} \\quad k=1,\\dots,n,\n",
|
||
"\\label{_auto3} \\tag{4}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e22879fc",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"where each $a_{1k}^{(1)}$ is equal to the original $a_{1k}$ element. The other coefficients are"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3b765484",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto4\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
"a_{jk}^{(2)} = a_{jk}^{(1)}-\\frac{a_{j1}^{(1)}a_{1k}^{(1)}}{a_{11}^{(1)}} \\quad j,k=2,\\dots,n,\n",
|
||
"\\label{_auto4} \\tag{5}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9383783c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"with a new right-hand side given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c0a3994f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto5\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
"y_{1}=w_1^{(1)}, \\quad w_j^{(2)} =w_j^{(1)}-\\frac{a_{j1}^{(1)}w_1^{(1)}}{a_{11}^{(1)}} \\quad j=2,\\dots,n.\n",
|
||
"\\label{_auto5} \\tag{6}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b5fab15e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We have also set $w_1^{(1)}=w_1$, the original vector element.\n",
|
||
"We see that the system of unknowns $x_1,\\dots,x_n$ is transformed into an $(n-1)\\times (n-1)$ problem.\n",
|
||
"\n",
|
||
"This step is called forward substitution.\n",
|
||
"Proceeding with these substitutions, we obtain the\n",
|
||
"general expressions for the new coefficients"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4d2d01dd",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto6\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" a_{jk}^{(m+1)} = a_{jk}^{(m)}-\\frac{a_{jm}^{(m)}a_{mk}^{(m)}}{a_{mm}^{(m)}} \\quad j,k=m+1,\\dots,n,\n",
|
||
"\\label{_auto6} \\tag{7}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8c1e5ff9",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"with $m=1,\\dots,n-1$ and a\n",
|
||
"right-hand side given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7165e693",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto7\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" w_j^{(m+1)} =w_j^{(m)}-\\frac{a_{jm}^{(m)}w_m^{(m)}}{a_{mm}^{(m)}}\\quad j=m+1,\\dots,n.\n",
|
||
"\\label{_auto7} \\tag{8}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c6646d7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"This set of $n-1$ elimations leads us to an equations which is solved by back substitution.\n",
|
||
"If the arithmetics is exact and the matrix $\\mathbf{A}$ is not singular, then the computed answer will be exact.\n",
|
||
"\n",
|
||
"Even though the matrix elements along the diagonal are not zero,\n",
|
||
"numerically small numbers may appear and subsequent divisions may lead to large numbers, which, if added\n",
|
||
"to a small number may yield losses of precision. Suppose for example that our first division in $(a_{22}-a_{21}a_{12}/a_{11})$\n",
|
||
"results in $-10^{-7}$ and that $a_{22}$ is one.\n",
|
||
"one. We are then\n",
|
||
"adding $10^7+1$. With single precision this results in $10^7$.\n",
|
||
"\n",
|
||
" * Gaussian elimination, $O(2/3n^3)$ flops, general matrix\n",
|
||
"\n",
|
||
" * LU decomposition, upper triangular and lower tridiagonal matrices, $O(2/3n^3)$ flops, general matrix. Get easily the inverse, determinant and can solve linear equations with back-substitution only, $O(n^2)$ flops\n",
|
||
"\n",
|
||
" * Cholesky decomposition. Real symmetric or hermitian positive definite matrix, $O(1/3n^3)$ flops.\n",
|
||
"\n",
|
||
" * Tridiagonal linear systems, important for differential equations. Normally positive definite and non-singular. $O(8n)$ flops for symmetric. Special case of banded matrices.\n",
|
||
"\n",
|
||
" * Singular value decomposition\n",
|
||
"\n",
|
||
" * the QR method will be discussed in chapter 7 in connection with eigenvalue systems. $O(4/3n^3)$ flops.\n",
|
||
"\n",
|
||
"The LU decomposition method means that we can rewrite\n",
|
||
"this matrix as the product of two matrices $\\mathbf{L}$ and $\\mathbf{U}$\n",
|
||
"where"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cc3c0cf7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{bmatrix}\n",
|
||
" a_{11} & a_{12} & a_{13} & a_{14} \\\\\n",
|
||
" a_{21} & a_{22} & a_{23} & a_{24} \\\\\n",
|
||
" a_{31} & a_{32} & a_{33} & a_{34} \\\\\n",
|
||
" a_{41} & a_{42} & a_{43} & a_{44}\n",
|
||
" \\end{bmatrix}\n",
|
||
" = \\begin{bmatrix}\n",
|
||
" 1 & 0 & 0 & 0 \\\\\n",
|
||
" l_{21} & 1 & 0 & 0 \\\\\n",
|
||
" l_{31} & l_{32} & 1 & 0 \\\\\n",
|
||
" l_{41} & l_{42} & l_{43} & 1\n",
|
||
" \\end{bmatrix}\n",
|
||
" \\begin{bmatrix}\n",
|
||
" u_{11} & u_{12} & u_{13} & u_{14} \\\\\n",
|
||
" 0 & u_{22} & u_{23} & u_{24} \\\\\n",
|
||
" 0 & 0 & u_{33} & u_{34} \\\\\n",
|
||
" 0 & 0 & 0 & u_{44}\n",
|
||
" \\end{bmatrix}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7256c8d3",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"LU decomposition forms the backbone of other algorithms in linear algebra, such as the\n",
|
||
"solution of linear equations given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "634741e9",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{11}x_1 +a_{12}x_2 +a_{13}x_3 + a_{14}x_4=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8a16abbb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4=w_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "442b2dd1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4=w_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2be3e530",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8f743387",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The above set of equations is conveniently solved by using LU decomposition as an intermediate step.\n",
|
||
"\n",
|
||
"The matrix $\\mathbf{A}\\in \\mathbb{R}^{n\\times n}$ has an LU factorization if the determinant\n",
|
||
"is different from zero. If the LU factorization exists and $\\mathbf{A}$ is non-singular, then the LU factorization\n",
|
||
"is unique and the determinant is given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0fe06953",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"det\\{\\mathbf{A}\\}=det\\{\\mathbf{LU}\\}= det\\{\\mathbf{L}\\}det\\{\\mathbf{U}\\}=u_{11}u_{22}\\dots u_{nn}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8232172f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"There are at least three main advantages with LU decomposition compared with standard Gaussian elimination:\n",
|
||
"\n",
|
||
" * It is straightforward to compute the determinant of a matrix\n",
|
||
"\n",
|
||
" * If we have to solve sets of linear equations with the same matrix but with different vectors $\\mathbf{y}$, the number of FLOPS is of the order $n^3$.\n",
|
||
"\n",
|
||
" * The inverse is such an operation \n",
|
||
"\n",
|
||
"With the LU decomposition it is rather\n",
|
||
"simple to solve a system of linear equations"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "87c2e89a",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{11}x_1 +a_{12}x_2 +a_{13}x_3 + a_{14}x_4=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "50fc7a76",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4=w_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e0379f33",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4=w_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "11e67b7e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8cda9070",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"This can be written in matrix form as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "908f05f7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{Ax}=\\mathbf{w}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "53053832",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"where $\\mathbf{A}$ and $\\mathbf{w}$ are known and we have to solve for\n",
|
||
"$\\mathbf{x}$. Using the LU dcomposition we write"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f3314fa8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A} \\mathbf{x} \\equiv \\mathbf{L} \\mathbf{U} \\mathbf{x} =\\mathbf{w}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2043fafc",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The previous equation can be calculated in two steps"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6fff8406",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{L} \\mathbf{y} = \\mathbf{w};\\qquad \\mathbf{Ux}=\\mathbf{y}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9a60901d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"To show that this is correct we use to the LU decomposition\n",
|
||
"to rewrite our system of linear equations as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "daacf54d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LUx}=\\mathbf{w},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "da702290",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and since the determinant of $\\mathbf{L}$ is equal to 1 (by construction\n",
|
||
"since the diagonals of $\\mathbf{L}$ equal 1) we can use the inverse of\n",
|
||
"$\\mathbf{L}$ to obtain"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c2a04b00",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{Ux}=\\mathbf{L^{-1}w}=\\mathbf{y},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d1214f7f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"which yields the intermediate step"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e8572956",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{L^{-1}w}=\\mathbf{y}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6920b812",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and as soon as we have $\\mathbf{y}$ we can obtain $\\mathbf{x}$\n",
|
||
"through $\\mathbf{Ux}=\\mathbf{y}$.\n",
|
||
"\n",
|
||
"For our four-dimentional example this takes the form"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "379ca640",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"y_1=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8c753134",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"l_{21}y_1 + y_2=w_2\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "12e32410",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"l_{31}y_1 + l_{32}y_2 + y_3 =w_3\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "01e9e4ae",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"l_{41}y_1 + l_{42}y_2 + l_{43}y_3 + y_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a27d4632",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4dae7cfb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{11}x_1 +u_{12}x_2 +u_{13}x_3 + u_{14}x_4=y_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5241a055",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{22}x_2 + u_{23}x_3 + u_{24}x_4=y_2\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f590f30b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{33}x_3 + u_{34}x_4=y_3\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "dbddd25a",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{44}x_4=y_4 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8d5dc72b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"This example shows the basis for the algorithm\n",
|
||
"needed to solve the set of $n$ linear equations.\n",
|
||
"\n",
|
||
"The algorithm goes as follows\n",
|
||
"\n",
|
||
" * Set up the matrix $\\bf A$ and the vector $\\bf w$ with their correct dimensions. This determines the dimensionality of the unknown vector $\\bf x$.\n",
|
||
"\n",
|
||
" * Then LU decompose the matrix $\\bf A$ through a call to the function `ludcmp(double a, int n, int indx, double &d)`. This functions returns the LU decomposed matrix $\\bf A$, its determinant and the vector indx which keeps track of the number of interchanges of rows. If the determinant is zero, the solution is malconditioned.\n",
|
||
"\n",
|
||
" * Thereafter you call the function `lubksb(double a, int n, int indx, double w)` which uses the LU decomposed matrix $\\bf A$ and the vector $\\bf w$ and returns $\\bf x$ in the same place as $\\bf w$. Upon exit the original content in $\\bf w$ is destroyed. If you wish to keep this information, you should make a backup of it in your calling function."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "fcf615b7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"### LU Decomposition, the inverse of a matrix\n",
|
||
"\n",
|
||
"If the inverse exists then"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3e2be427",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}^{-1}\\mathbf{A}=\\mathbf{I},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "519b78bb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"the identity matrix. With an LU decomposed matrix we can rewrite the last equation as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a730fd5f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LU}\\mathbf{A}^{-1}=\\mathbf{I}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2cc4e115",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"If we assume that the first column (that is column 1) of the inverse matrix\n",
|
||
"can be written as a vector with unknown entries"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "03c4a0ae",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}_1^{-1}= \\begin{bmatrix}\n",
|
||
" a_{11}^{-1} \\\\\n",
|
||
" a_{21}^{-1} \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" a_{n1}^{-1} \\\\\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "93ef84cc",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"then we have a linear set of equations"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "04c367a7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LU}\\begin{bmatrix}\n",
|
||
" a_{11}^{-1} \\\\\n",
|
||
" a_{21}^{-1} \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" a_{n1}^{-1} \\\\\n",
|
||
" \\end{bmatrix} =\\begin{bmatrix}\n",
|
||
" 1 \\\\\n",
|
||
" 0 \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" 0 \\\\\n",
|
||
" \\end{bmatrix}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8f903ad8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"In a similar way we can compute the unknow entries of the second column,"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c343638b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LU}\\begin{bmatrix}\n",
|
||
" a_{12}^{-1} \\\\\n",
|
||
" a_{22}^{-1} \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" a_{n2}^{-1} \\\\\n",
|
||
" \\end{bmatrix}=\\begin{bmatrix}\n",
|
||
" 0 \\\\\n",
|
||
" 1 \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" 0 \\\\\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "1c013532",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and continue till we have solved all $n$ sets of linear equations.\n",
|
||
"\n",
|
||
"The calculation of the inverse here assumes that it actually\n",
|
||
"exists. In many machine learning applications there may be strong\n",
|
||
"linear dependencies among the various columns and/or rows. In our\n",
|
||
"discussions of linear regression we will dive into the mathematics of\n",
|
||
"the singular value decomposition, an algorithm which will allow us to calculate the so-called pseudo-inverse.\n",
|
||
"These details will be presented in our linear regression chapter."
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.9.10"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
} |