lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5c6c05efbe058f033f3f35def695f64a7c70b9ce
FYS-STK4155/doc/pub/odenn/html/._odenn-bs033.html
T
mhjensen 489e7cf8bf update on decision trees
2018-11-11 11:53:49 +01:00

535 lines
39 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Using Neural networks to solve ODEs and PDEs">
<title>Data Analysis and Machine Learning: Using Neural networks to solve ODEs and PDEs</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Differential equations', 2, None, '___sec0'),
('Description of the equation to solve for', 2, None, '___sec1'),
('Ordinary Differential Equations', 2, None, '___sec2'),
('The trial solution', 2, None, '___sec3'),
('Minimizing the cost function using gradient descent and '
'automatic differentiation',
2,
None,
'___sec4'),
('Example: Exponential decay and setting up the network using '
'Autograd',
2,
None,
'___sec5'),
('The function to solve for', 2, None, '___sec6'),
('The trial solution', 2, None, '___sec7'),
('Reformulating the problem', 2, None, '___sec8'),
('A possible implementation of a neural network using Autograd',
2,
None,
'___sec9'),
('Backpropagation using Autograd', 2, None, '___sec10'),
('Gradient descent', 2, None, '___sec11'),
('The network with one input, hidden, and output layer',
2,
None,
'___sec12'),
('The network with one input layer, specified number of hidden '
'layers, and one output layer output layer',
2,
None,
'___sec13'),
('Example: Population growth, comparing Autograd, TensorFlow, '
"and Euler's scheme",
2,
None,
'___sec14'),
('Setting up the problem', 2, None, '___sec15'),
('The trial solution', 2, None, '___sec16'),
('The program using Autograd', 2, None, '___sec17'),
('Using forward Euler to solve the ODE', 2, None, '___sec18'),
('Using TensorFlow to model logistic population growth',
2,
None,
'___sec19'),
('The general program flow in TensorFlow', 2, None, '___sec20'),
('Program flow in TensorFlow - Construction phase',
2,
None,
'___sec21'),
('Program flow in TensorFlow - Execution phase',
2,
None,
'___sec22'),
('The full program modeling logistic population growth using '
'TensorFlow',
2,
None,
'___sec23'),
('Example: Solving the one dimensional Poisson equation using '
'Autograd and TensorFlow',
2,
None,
'___sec24'),
('The specific equation to solve for', 2, None, '___sec25'),
('Solving the equation using Autograd', 2, None, '___sec26'),
('Comparing with a numerical scheme', 2, None, '___sec27'),
('Using gradient descent in TensorFlow to solve Poisson equation',
2,
None,
'___sec28'),
('Using a different optimization algorithm implemented in '
'TensorFlow to solve Poisson equation',
2,
None,
'___sec29'),
('Partial Differential Equations', 2, None, '___sec30'),
('Example: The diffusion equation', 2, None, '___sec31'),
('Defining the problem', 2, None, '___sec32'),
('Setting up the network using Autograd', 2, None, '___sec33'),
('Setting up the network using Autograd; The trial solution',
2,
None,
'___sec34'),
('Setting up the network using Autograd; The full program',
2,
None,
'___sec35'),
('Example: Solving the wave equation using Autograd and '
'TensorFlow',
2,
None,
'___sec36'),
('The problem to solve for', 2, None, '___sec37'),
('The trial solution', 2, None, '___sec38'),
('The analytical solution', 2, None, '___sec39'),
('Solving the wave equation - the full program using Autograd',
2,
None,
'___sec40'),
('Solving the wave equation - the full program using TensorFlow',
2,
None,
'___sec41'),
('Resources', 2, None, '___sec42')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="odenn-bs.html">Data Analysis and Machine Learning: Using Neural networks to solve ODEs and PDEs</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._odenn-bs001.html#___sec0" style="font-size: 80%;">Differential equations</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs002.html#___sec1" style="font-size: 80%;">Description of the equation to solve for</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs003.html#___sec2" style="font-size: 80%;">Ordinary Differential Equations</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs004.html#___sec3" style="font-size: 80%;">The trial solution</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs005.html#___sec4" style="font-size: 80%;">Minimizing the cost function using gradient descent and automatic differentiation</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs006.html#___sec5" style="font-size: 80%;">Example: Exponential decay and setting up the network using Autograd</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs007.html#___sec6" style="font-size: 80%;">The function to solve for</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs008.html#___sec7" style="font-size: 80%;">The trial solution</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs009.html#___sec8" style="font-size: 80%;">Reformulating the problem</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs010.html#___sec9" style="font-size: 80%;">A possible implementation of a neural network using Autograd</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs011.html#___sec10" style="font-size: 80%;">Backpropagation using Autograd</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs012.html#___sec11" style="font-size: 80%;">Gradient descent</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs013.html#___sec12" style="font-size: 80%;">The network with one input, hidden, and output layer</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs014.html#___sec13" style="font-size: 80%;">The network with one input layer, specified number of hidden layers, and one output layer output layer</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs015.html#___sec14" style="font-size: 80%;">Example: Population growth, comparing Autograd, TensorFlow, and Euler's scheme</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs016.html#___sec15" style="font-size: 80%;">Setting up the problem</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs017.html#___sec16" style="font-size: 80%;">The trial solution</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs018.html#___sec17" style="font-size: 80%;">The program using Autograd</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs019.html#___sec18" style="font-size: 80%;">Using forward Euler to solve the ODE</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs020.html#___sec19" style="font-size: 80%;">Using TensorFlow to model logistic population growth</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs021.html#___sec20" style="font-size: 80%;">The general program flow in TensorFlow</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs021.html#___sec21" style="font-size: 80%;">Program flow in TensorFlow - Construction phase</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs021.html#___sec22" style="font-size: 80%;">Program flow in TensorFlow - Execution phase</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs021.html#___sec23" style="font-size: 80%;">The full program modeling logistic population growth using TensorFlow</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs022.html#___sec24" style="font-size: 80%;">Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs023.html#___sec25" style="font-size: 80%;">The specific equation to solve for</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs024.html#___sec26" style="font-size: 80%;">Solving the equation using Autograd</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs025.html#___sec27" style="font-size: 80%;">Comparing with a numerical scheme</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs026.html#___sec28" style="font-size: 80%;">Using gradient descent in TensorFlow to solve Poisson equation</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs027.html#___sec29" style="font-size: 80%;">Using a different optimization algorithm implemented in TensorFlow to solve Poisson equation</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs028.html#___sec30" style="font-size: 80%;">Partial Differential Equations</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs029.html#___sec31" style="font-size: 80%;">Example: The diffusion equation</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs030.html#___sec32" style="font-size: 80%;">Defining the problem</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs031.html#___sec33" style="font-size: 80%;">Setting up the network using Autograd</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs032.html#___sec34" style="font-size: 80%;">Setting up the network using Autograd; The trial solution</a></li>
<!-- navigation toc: --> <li><a href="#___sec35" style="font-size: 80%;">Setting up the network using Autograd; The full program</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs034.html#___sec36" style="font-size: 80%;">Example: Solving the wave equation using Autograd and TensorFlow</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs035.html#___sec37" style="font-size: 80%;">The problem to solve for</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs036.html#___sec38" style="font-size: 80%;">The trial solution</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs037.html#___sec39" style="font-size: 80%;">The analytical solution</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs038.html#___sec40" style="font-size: 80%;">Solving the wave equation - the full program using Autograd</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs039.html#___sec41" style="font-size: 80%;">Solving the wave equation - the full program using TensorFlow</a></li>
<!-- navigation toc: --> <li><a href="._odenn-bs040.html#___sec42" style="font-size: 80%;">Resources</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0033"></a>
<!-- !split -->
<h2 id="___sec35" class="anchor">Setting up the network using Autograd; The full program </h2>
Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution.
<p>
The analytical solution of our problem is
$$
g(x,t) = \exp(-\pi^2 t)\sin(\pi x)
$$
<p>
A possible way to implement a neural network solving the PDE, is given below.
Be aware, though, that it is fairly slow for the parameters used.
A better result is possible, but requires more iterations, and thus longer time to complete.
<p>
Using only 20 neurons in one hidden layer, the program managed to make the trial solution have the maximum absolute error of 0.0075. The execution time, however, was approximately one day and 14 hours on a computer having Intel i7-7560U 2.4 GHz CPU.
<p>
Indeed, the program below is not optimal in its implementation, but rather serves as an example on how to implement and use a neural network to solve a PDE.
Using TensorFlow in the next example sovling the wave equation, has a much better execution time.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">autograd.numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">autograd</span> <span style="color: #008000; font-weight: bold">import</span> jacobian,hessian,grad
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">autograd.numpy.random</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">npr</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">matplotlib</span> <span style="color: #008000; font-weight: bold">import</span> cm
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">matplotlib</span> <span style="color: #008000; font-weight: bold">import</span> pyplot <span style="color: #008000; font-weight: bold">as</span> plt
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">mpl_toolkits.mplot3d</span> <span style="color: #008000; font-weight: bold">import</span> axes3d
<span style="color: #408080; font-style: italic">## Set up the network</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">sigmoid</span>(z):
<span style="color: #008000; font-weight: bold">return</span> <span style="color: #666666">1/</span>(<span style="color: #666666">1</span> <span style="color: #666666">+</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>z))
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">deep_neural_network</span>(deep_params, x):
<span style="color: #408080; font-style: italic"># x is now a point and a 1D numpy array; make it a column vector</span>
num_coordinates <span style="color: #666666">=</span> np<span style="color: #666666">.</span>size(x,<span style="color: #666666">0</span>)
x <span style="color: #666666">=</span> x<span style="color: #666666">.</span>reshape(num_coordinates,<span style="color: #666666">-1</span>)
num_points <span style="color: #666666">=</span> np<span style="color: #666666">.</span>size(x,<span style="color: #666666">1</span>)
<span style="color: #408080; font-style: italic"># N_hidden is the number of hidden layers</span>
N_hidden <span style="color: #666666">=</span> np<span style="color: #666666">.</span>size(deep_params) <span style="color: #666666">-</span> <span style="color: #666666">1</span> <span style="color: #408080; font-style: italic"># -1 since params consist of parameters to all the hidden layers AND the output layer</span>
<span style="color: #408080; font-style: italic"># Assume that the input layer does nothing to the input x</span>
x_input <span style="color: #666666">=</span> x
x_prev <span style="color: #666666">=</span> x_input
<span style="color: #408080; font-style: italic">## Hidden layers:</span>
<span style="color: #008000; font-weight: bold">for</span> l <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(N_hidden):
<span style="color: #408080; font-style: italic"># From the list of parameters P; find the correct weigths and bias for this layer</span>
w_hidden <span style="color: #666666">=</span> deep_params[l]
<span style="color: #408080; font-style: italic"># Add a row of ones to include bias</span>
x_prev <span style="color: #666666">=</span> np<span style="color: #666666">.</span>concatenate((np<span style="color: #666666">.</span>ones((<span style="color: #666666">1</span>,num_points)), x_prev ), axis <span style="color: #666666">=</span> <span style="color: #666666">0</span>)
z_hidden <span style="color: #666666">=</span> np<span style="color: #666666">.</span>matmul(w_hidden, x_prev)
x_hidden <span style="color: #666666">=</span> sigmoid(z_hidden)
<span style="color: #408080; font-style: italic"># Update x_prev such that next layer can use the output from this layer</span>
x_prev <span style="color: #666666">=</span> x_hidden
<span style="color: #408080; font-style: italic">## Output layer:</span>
<span style="color: #408080; font-style: italic"># Get the weights and bias for this layer</span>
w_output <span style="color: #666666">=</span> deep_params[<span style="color: #666666">-1</span>]
<span style="color: #408080; font-style: italic"># Include bias:</span>
x_prev <span style="color: #666666">=</span> np<span style="color: #666666">.</span>concatenate((np<span style="color: #666666">.</span>ones((<span style="color: #666666">1</span>,num_points)), x_prev), axis <span style="color: #666666">=</span> <span style="color: #666666">0</span>)
z_output <span style="color: #666666">=</span> np<span style="color: #666666">.</span>matmul(w_output, x_prev)
x_output <span style="color: #666666">=</span> z_output
<span style="color: #008000; font-weight: bold">return</span> x_output[<span style="color: #666666">0</span>][<span style="color: #666666">0</span>]
<span style="color: #408080; font-style: italic">## Define the trial solution and cost function</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">u</span>(x):
<span style="color: #008000; font-weight: bold">return</span> np<span style="color: #666666">.</span>sin(np<span style="color: #666666">.</span>pi<span style="color: #666666">*</span>x)
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">g_trial</span>(point,P):
x,t <span style="color: #666666">=</span> point
<span style="color: #008000; font-weight: bold">return</span> (<span style="color: #666666">1-</span>t)<span style="color: #666666">*</span>u(x) <span style="color: #666666">+</span> x<span style="color: #666666">*</span>(<span style="color: #666666">1-</span>x)<span style="color: #666666">*</span>t<span style="color: #666666">*</span>deep_neural_network(P,point)
<span style="color: #408080; font-style: italic"># The right side of the ODE:</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">f</span>(point):
<span style="color: #008000; font-weight: bold">return</span> <span style="color: #666666">0.</span>
<span style="color: #408080; font-style: italic"># The cost function:</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">cost_function</span>(P, x, t):
cost_sum <span style="color: #666666">=</span> <span style="color: #666666">0</span>
g_t_jacobian_func <span style="color: #666666">=</span> jacobian(g_trial)
g_t_hessian_func <span style="color: #666666">=</span> hessian(g_trial)
<span style="color: #008000; font-weight: bold">for</span> x_ <span style="color: #AA22FF; font-weight: bold">in</span> x:
<span style="color: #008000; font-weight: bold">for</span> t_ <span style="color: #AA22FF; font-weight: bold">in</span> t:
point <span style="color: #666666">=</span> np<span style="color: #666666">.</span>array([x_,t_])
g_t <span style="color: #666666">=</span> g_trial(point,P)
g_t_jacobian <span style="color: #666666">=</span> g_t_jacobian_func(point,P)
g_t_hessian <span style="color: #666666">=</span> g_t_hessian_func(point,P)
g_t_dt <span style="color: #666666">=</span> g_t_jacobian[<span style="color: #666666">1</span>]
g_t_d2x <span style="color: #666666">=</span> g_t_hessian[<span style="color: #666666">0</span>][<span style="color: #666666">0</span>]
func <span style="color: #666666">=</span> f(point)
err_sqr <span style="color: #666666">=</span> ( (g_t_dt <span style="color: #666666">-</span> g_t_d2x) <span style="color: #666666">-</span> func)<span style="color: #666666">**2</span>
cost_sum <span style="color: #666666">+=</span> err_sqr
<span style="color: #008000; font-weight: bold">return</span> cost_sum <span style="color: #666666">/</span>( np<span style="color: #666666">.</span>size(x)<span style="color: #666666">*</span>np<span style="color: #666666">.</span>size(t) )
<span style="color: #408080; font-style: italic">## For comparison, define the analytical solution</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">g_analytic</span>(point):
x,t <span style="color: #666666">=</span> point
<span style="color: #008000; font-weight: bold">return</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>np<span style="color: #666666">.</span>pi<span style="color: #666666">**2*</span>t)<span style="color: #666666">*</span>np<span style="color: #666666">.</span>sin(np<span style="color: #666666">.</span>pi<span style="color: #666666">*</span>x)
<span style="color: #408080; font-style: italic">## Set up a function for training the network to solve for the equation</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">solve_pde_deep_neural_network</span>(x,t, num_neurons, num_iter, lmb):
<span style="color: #408080; font-style: italic">## Set up initial weigths and biases</span>
N_hidden <span style="color: #666666">=</span> np<span style="color: #666666">.</span>size(num_neurons)
<span style="color: #408080; font-style: italic">## Set up initial weigths and biases</span>
<span style="color: #408080; font-style: italic"># Initialize the list of parameters:</span>
P <span style="color: #666666">=</span> [<span style="color: #008000">None</span>]<span style="color: #666666">*</span>(N_hidden <span style="color: #666666">+</span> <span style="color: #666666">1</span>) <span style="color: #408080; font-style: italic"># + 1 to include the output layer</span>
P[<span style="color: #666666">0</span>] <span style="color: #666666">=</span> npr<span style="color: #666666">.</span>randn(num_neurons[<span style="color: #666666">0</span>], <span style="color: #666666">2</span> <span style="color: #666666">+</span> <span style="color: #666666">1</span> ) <span style="color: #408080; font-style: italic"># 2 since we have two points, +1 to include bias</span>
<span style="color: #008000; font-weight: bold">for</span> l <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">1</span>,N_hidden):
P[l] <span style="color: #666666">=</span> npr<span style="color: #666666">.</span>randn(num_neurons[l], num_neurons[l<span style="color: #666666">-1</span>] <span style="color: #666666">+</span> <span style="color: #666666">1</span>) <span style="color: #408080; font-style: italic"># +1 to include bias</span>
<span style="color: #408080; font-style: italic"># For the output layer</span>
P[<span style="color: #666666">-1</span>] <span style="color: #666666">=</span> npr<span style="color: #666666">.</span>randn(<span style="color: #666666">1</span>, num_neurons[<span style="color: #666666">-1</span>] <span style="color: #666666">+</span> <span style="color: #666666">1</span> ) <span style="color: #408080; font-style: italic"># +1 since bias is included</span>
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&#39;Initial cost: &#39;</span>,cost_function(P, x, t))
cost_function_grad <span style="color: #666666">=</span> grad(cost_function,<span style="color: #666666">0</span>)
<span style="color: #408080; font-style: italic"># Let the update be done num_iter times</span>
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(num_iter):
cost_grad <span style="color: #666666">=</span> cost_function_grad(P, x , t)
<span style="color: #008000; font-weight: bold">for</span> l <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(N_hidden<span style="color: #666666">+1</span>):
P[l] <span style="color: #666666">=</span> P[l] <span style="color: #666666">-</span> lmb <span style="color: #666666">*</span> cost_grad[l]
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&#39;Final cost: &#39;</span>,cost_function(P, x, t))
<span style="color: #008000; font-weight: bold">return</span> P
<span style="color: #008000; font-weight: bold">if</span> <span style="color: #19177C">__name__</span> <span style="color: #666666">==</span> <span style="color: #BA2121">&#39;__main__&#39;</span>:
<span style="color: #408080; font-style: italic">### Use the neural network:</span>
npr<span style="color: #666666">.</span>seed(<span style="color: #666666">15</span>)
<span style="color: #408080; font-style: italic">## Decide the vales of arguments to the function to solve</span>
Nx <span style="color: #666666">=</span> <span style="color: #666666">10</span>; Nt <span style="color: #666666">=</span> <span style="color: #666666">10</span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>, <span style="color: #666666">1</span>, Nx)
t <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>,<span style="color: #666666">1</span>,Nt)
<span style="color: #408080; font-style: italic">## Set up the parameters for the network</span>
num_hidden_neurons <span style="color: #666666">=</span> [<span style="color: #666666">100</span>, <span style="color: #666666">25</span>]
num_iter <span style="color: #666666">=</span> <span style="color: #666666">250</span>
lmb <span style="color: #666666">=</span> <span style="color: #666666">0.01</span>
P <span style="color: #666666">=</span> solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)
<span style="color: #408080; font-style: italic">## Store the results</span>
g_dnn_ag <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros((Nx, Nt))
G_analytical <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros((Nx, Nt))
<span style="color: #008000; font-weight: bold">for</span> i,x_ <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">enumerate</span>(x):
<span style="color: #008000; font-weight: bold">for</span> j, t_ <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">enumerate</span>(t):
point <span style="color: #666666">=</span> np<span style="color: #666666">.</span>array([x_, t_])
g_dnn_ag[i,j] <span style="color: #666666">=</span> g_trial(point,P)
G_analytical[i,j] <span style="color: #666666">=</span> g_analytic(point)
<span style="color: #408080; font-style: italic"># Find the map difference between the analytical and the computed solution</span>
diff_ag <span style="color: #666666">=</span> np<span style="color: #666666">.</span>abs(g_dnn_ag <span style="color: #666666">-</span> G_analytical)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&#39;Max absolute difference between the analytical solution and the network: </span><span style="color: #BB6688; font-weight: bold">%g</span><span style="color: #BA2121">&#39;</span><span style="color: #666666">%</span>np<span style="color: #666666">.</span>max(diff_ag))
<span style="color: #408080; font-style: italic">## Plot the solutions in two dimensions, that being in position and time</span>
T,X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>meshgrid(t,x)
fig <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>figure(figsize<span style="color: #666666">=</span>(<span style="color: #666666">10</span>,<span style="color: #666666">10</span>))
ax <span style="color: #666666">=</span> fig<span style="color: #666666">.</span>gca(projection<span style="color: #666666">=</span><span style="color: #BA2121">&#39;3d&#39;</span>)
ax<span style="color: #666666">.</span>set_title(<span style="color: #BA2121">&#39;Solution from the deep neural network w/ </span><span style="color: #BB6688; font-weight: bold">%d</span><span style="color: #BA2121"> layer&#39;</span><span style="color: #666666">%</span><span style="color: #008000">len</span>(num_hidden_neurons))
s <span style="color: #666666">=</span> ax<span style="color: #666666">.</span>plot_surface(T,X,g_dnn_ag,linewidth<span style="color: #666666">=0</span>,antialiased<span style="color: #666666">=</span><span style="color: #008000">False</span>,cmap<span style="color: #666666">=</span>cm<span style="color: #666666">.</span>viridis)
ax<span style="color: #666666">.</span>set_xlabel(<span style="color: #BA2121">&#39;Time $t$&#39;</span>)
ax<span style="color: #666666">.</span>set_ylabel(<span style="color: #BA2121">&#39;Position $x$&#39;</span>);
fig <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>figure(figsize<span style="color: #666666">=</span>(<span style="color: #666666">10</span>,<span style="color: #666666">10</span>))
ax <span style="color: #666666">=</span> fig<span style="color: #666666">.</span>gca(projection<span style="color: #666666">=</span><span style="color: #BA2121">&#39;3d&#39;</span>)
ax<span style="color: #666666">.</span>set_title(<span style="color: #BA2121">&#39;Analytical solution&#39;</span>)
s <span style="color: #666666">=</span> ax<span style="color: #666666">.</span>plot_surface(T,X,G_analytical,linewidth<span style="color: #666666">=0</span>,antialiased<span style="color: #666666">=</span><span style="color: #008000">False</span>,cmap<span style="color: #666666">=</span>cm<span style="color: #666666">.</span>viridis)
ax<span style="color: #666666">.</span>set_xlabel(<span style="color: #BA2121">&#39;Time $t$&#39;</span>)
ax<span style="color: #666666">.</span>set_ylabel(<span style="color: #BA2121">&#39;Position $x$&#39;</span>);
fig <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>figure(figsize<span style="color: #666666">=</span>(<span style="color: #666666">10</span>,<span style="color: #666666">10</span>))
ax <span style="color: #666666">=</span> fig<span style="color: #666666">.</span>gca(projection<span style="color: #666666">=</span><span style="color: #BA2121">&#39;3d&#39;</span>)
ax<span style="color: #666666">.</span>set_title(<span style="color: #BA2121">&#39;Difference&#39;</span>)
s <span style="color: #666666">=</span> ax<span style="color: #666666">.</span>plot_surface(T,X,diff_ag,linewidth<span style="color: #666666">=0</span>,antialiased<span style="color: #666666">=</span><span style="color: #008000">False</span>,cmap<span style="color: #666666">=</span>cm<span style="color: #666666">.</span>viridis)
ax<span style="color: #666666">.</span>set_xlabel(<span style="color: #BA2121">&#39;Time $t$&#39;</span>)
ax<span style="color: #666666">.</span>set_ylabel(<span style="color: #BA2121">&#39;Position $x$&#39;</span>);
<span style="color: #408080; font-style: italic">## Take some slices of the 3D plots just to see the solutions at particular times</span>
indx1 <span style="color: #666666">=</span> <span style="color: #666666">0</span>
indx2 <span style="color: #666666">=</span> <span style="color: #008000">int</span>(Nt<span style="color: #666666">/2</span>)
indx3 <span style="color: #666666">=</span> Nt<span style="color: #666666">-1</span>
t1 <span style="color: #666666">=</span> t[indx1]
t2 <span style="color: #666666">=</span> t[indx2]
t3 <span style="color: #666666">=</span> t[indx3]
<span style="color: #408080; font-style: italic"># Slice the results from the DNN</span>
res1 <span style="color: #666666">=</span> g_dnn_ag[:,indx1]
res2 <span style="color: #666666">=</span> g_dnn_ag[:,indx2]
res3 <span style="color: #666666">=</span> g_dnn_ag[:,indx3]
<span style="color: #408080; font-style: italic"># Slice the analytical results</span>
res_analytical1 <span style="color: #666666">=</span> G_analytical[:,indx1]
res_analytical2 <span style="color: #666666">=</span> G_analytical[:,indx2]
res_analytical3 <span style="color: #666666">=</span> G_analytical[:,indx3]
<span style="color: #408080; font-style: italic"># Plot the slices</span>
plt<span style="color: #666666">.</span>figure(figsize<span style="color: #666666">=</span>(<span style="color: #666666">10</span>,<span style="color: #666666">10</span>))
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Computed solutions at time = </span><span style="color: #BB6688; font-weight: bold">%g</span><span style="color: #BA2121">&quot;</span><span style="color: #666666">%</span>t1)
plt<span style="color: #666666">.</span>plot(x, res1)
plt<span style="color: #666666">.</span>plot(x,res_analytical1)
plt<span style="color: #666666">.</span>legend([<span style="color: #BA2121">&#39;dnn&#39;</span>,<span style="color: #BA2121">&#39;analytical&#39;</span>])
plt<span style="color: #666666">.</span>figure(figsize<span style="color: #666666">=</span>(<span style="color: #666666">10</span>,<span style="color: #666666">10</span>))
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Computed solutions at time = </span><span style="color: #BB6688; font-weight: bold">%g</span><span style="color: #BA2121">&quot;</span><span style="color: #666666">%</span>t2)
plt<span style="color: #666666">.</span>plot(x, res2)
plt<span style="color: #666666">.</span>plot(x,res_analytical2)
plt<span style="color: #666666">.</span>legend([<span style="color: #BA2121">&#39;dnn&#39;</span>,<span style="color: #BA2121">&#39;analytical&#39;</span>])
plt<span style="color: #666666">.</span>figure(figsize<span style="color: #666666">=</span>(<span style="color: #666666">10</span>,<span style="color: #666666">10</span>))
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">&quot;Computed solutions at time = </span><span style="color: #BB6688; font-weight: bold">%g</span><span style="color: #BA2121">&quot;</span><span style="color: #666666">%</span>t3)
plt<span style="color: #666666">.</span>plot(x, res3)
plt<span style="color: #666666">.</span>plot(x,res_analytical3)
plt<span style="color: #666666">.</span>legend([<span style="color: #BA2121">&#39;dnn&#39;</span>,<span style="color: #BA2121">&#39;analytical&#39;</span>])
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._odenn-bs032.html">&laquo;</a></li>
<li><a href="._odenn-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._odenn-bs025.html">26</a></li>
<li><a href="._odenn-bs026.html">27</a></li>
<li><a href="._odenn-bs027.html">28</a></li>
<li><a href="._odenn-bs028.html">29</a></li>
<li><a href="._odenn-bs029.html">30</a></li>
<li><a href="._odenn-bs030.html">31</a></li>
<li><a href="._odenn-bs031.html">32</a></li>
<li><a href="._odenn-bs032.html">33</a></li>
<li class="active"><a href="._odenn-bs033.html">34</a></li>
<li><a href="._odenn-bs034.html">35</a></li>
<li><a href="._odenn-bs035.html">36</a></li>
<li><a href="._odenn-bs036.html">37</a></li>
<li><a href="._odenn-bs037.html">38</a></li>
<li><a href="._odenn-bs038.html">39</a></li>
<li><a href="._odenn-bs039.html">40</a></li>
<li><a href="._odenn-bs040.html">41</a></li>
<li><a href="._odenn-bs034.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink