Chapter 2 Introduction to Neural network
Chapter 2 Introduction to Neural network
Chapter 2 Introduction to Neural network
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
¯W [3] =<br />
(rand(p, k + 1) − 0.5)<br />
k + 1<br />
where ”rand” is uniformly distributed in [0, 1].<br />
Step 2. Pick a input-target pair randomly from the training set.<br />
say (x i , t i ). Calculate the output when x i is the input according <strong>to</strong><br />
[<br />
[<br />
y i = H ¯W [3] · G[<br />
¯W [2] · F ¯W [1]¯x ]] ]<br />
i<br />
where<br />
F = [f 1 (u 1 ) [1] , f 2 (u 2 ) [1] , · · · , f m (u m ) [1] ] T<br />
G = [g 1 (u 1 ) [2] , g 2 (u 2 ) [2] , · · · , g k (u k ) [2] ] T<br />
H = [h 1 (u 1 ) [3] , h 2 (u 2 ) [3] , · · · , h p (u p ) [3] ] T<br />
All functions are chosen in advance.<br />
Step 3.<br />
Find the weight corrections for each layer.<br />
First define the vec<strong>to</strong>r e i = t i − y i .<br />
Define the local error vec<strong>to</strong>r δ [s] , s = 1, 2, 3<br />
δ [3] = diag(e) ∂H<br />
∂u [3]<br />
△ ¯W [3] = αδ [3] (ō [2] ) T<br />
, α − stepsize<br />
OBS! ō [2] is the extended vec<strong>to</strong>r!<br />
( (W<br />
δ [2] [3]<br />
= diag<br />
) )<br />
T<br />
δ<br />
[3] ∂G<br />
OBS! W [3] is without the biases!<br />
△ ¯W [2] = αδ [2] (ō [1] ) T<br />
∂u [2]<br />
( (W<br />
δ [1] [2]<br />
= diag<br />
) )<br />
T<br />
δ<br />
[2] ∂F<br />
∂u [1]<br />
△ ¯W [1] = αδ [1]¯x T 46
Change language