Chapter 2 Introduction to Neural network

More documents

Recommendations

Info

x w y ¡¢£¤ and define the columns of a identity matrix I m×m = [e 1 e 2 · · · e m ] i.e. ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 0 0 e 1 = ⎢ ⎥ ⎣ 0. ⎦ , e 2 = ⎢ ⎥ ⎣ 1. ⎦ , . . . , e m = ⎢ ⎥ ⎣ 0. ⎦ 0 0 1 then the training set is (a 1 , e 1 ), (a 2 , e 2 ), · · · , (a m , e m ) we use the LMS algorithm to adapt the weights W to the training set with one modification, that is W (k+1) = (1 − γ)W k + αex T γ ≪ 1- leaky factor, α - stepsize, x k =input=a k and e = error vector (remember we have linear functions, the algorithm is the MIMO- LMS, Multiple-Input Multiple-Output) After convergence W T = A + (Pseudoinverse of A) □ 6.5.1 One more example of Backpropagation An alternative way to find the pseudo inverse of a matrix A. We will use a linear two layer network. We create a random sequence of input vectors and use the same vectors as targets. ⇒ Training sequence(x 1 , x 1 ), (x 2 , x 2 ), . . . , (x N , x N ) We fix the first layer weights to A and we only update the output layer weights W. 51
¤ A ¤ W ¢£ Since we know that for any input vector x ¡ ¡ i ¢£ we have ˆx i = WAx i and that backpropagation minimizes the norm of min ‖ x i − WAx i ‖ ⇒ W = A + 6.6 Application speech synthesis-NETtalk People in the field of linguistics have been working with rule based speech synthesis systems for many years. Speech synthesis is away of converting text to speech and a feedforward neural network, NETtalk, can be trained to solve this. Is uses a 7 character sliding window to read the text and it is trained with backpropagation to classify the phonemes of the text’s pronunciation (See figure 9.14). Phonemes 26 output units 80 hidden units u r a l _ N e 7 groups of 29 input sites Another application is the reverse problem of listening to speech and then convert to text. They use Hidden Markov Models trained with Backpropagation. 6.7 Improvements to backpropagation There are many ways to improve the convergence of backpropagation. 52
Page 1 and 2: Chapter 2 Introduction to Neural ne
Page 3 and 4: ¡¢£¤¡¢£¥ ¡ ¢£¦ § ¡¢
Page 5 and 6: 2.5 Learning Depending on the task
Page 7 and 8: 3.1 Hadamard-Walsh transform For bi
Page 9 and 10: ¢£¤¥¦§ £ ¤¥¦¨ ¢¥¢¦
Page 11 and 12: ¡ ¢£ ¤¡¤¢¤£¥¦w x 4.2 The
Page 13 and 14: □ A formula for R(m, n) ∑n−1
Page 15 and 16: Assume that we have a sequence of i
Page 17 and 18: 6.1.2 The unipolar sigmoid function
Page 19 and 20: ¯W [3] = (rand(p, k + 1) − 0.5)
Page 21 and 22: where u [3] j = p∑ i=1 w [3] ji o
Page 23: 6.4 The rule of the hidden layer Co
Page 27 and 28: The backpropagation algorithm can c
Page 29 and 30: x - training data o - generalizatio
Page 31 and 32: 6.13.3 Cross validation A standard

Chapter 2 Introduction to Neural network

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?