Thinking-data-science-a-data-science-practitioners-guide

xviii
Contents
Part III ANN: Overview
17 Artificial Neural Networks ............................... 261
AI Evolution ........................................... 261
Artificial Neural Networks . . . .............................. 262
Perceptron .......................................... 262
What Is ANN? . . . .................................... 263
Network Training . . . . . ................................ 264
ANN Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
What Is DNN? . . . .................................... 264
Network Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
What Are Pre-trained Models? ............................ 265
Important Terms to Know ................................. 266
Activation Functions . .................................. 266
Back Propagation . .................................... 268
Vanishing and Exploding Gradients . ....................... 268
Optimization Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Types of Optimizers ................................... 270
Loss Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Types of Network Architectures . . ........................... 272
Convolutional Neural Network ............................. 272
Convolutional Layer . . ................................. 273
Pooling Layer ........................................ 273
Fully Connected Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
CNN Applications .................................... 274
Generative Adversarial Network . . . ......................... 274
Model Architecture .................................... 275
The Generator ....................................... 275
The Discriminator . . . .................................. 276
How Does GAN Work? . . . ............................. 276
How Data Scientists Use GAN? ........................... 277
Recurrent Neural Networks (RNN) . . . ........................ 278
Long Short-Term Memory (LSTM) . . . ....................... 279
Forget Gate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
Input Gate . . . . . . . . . . ................................ 280
Update Gate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Output Gate . . ....................................... 281
LSTM Applications ................................... 282
Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Pre-trained Models for Text . ............................. 283
Pre-trained Models for Image Data . ....................... 285
Advantages/Disadvantages ................................. 286
Summary ............................................. 286