Iterative Methods for Sparse Linear Systems Second Edition

More documents

Recommendations

Info

xx PREFACE Apart from two recent volumes by Axelsson [14] and Hackbusch [163], few books on iterative methods have appeared since the excellent ones by Varga [293]. and later Young [322]. Since then, researchers and practitioners have achieved remarkable progress in the development and use of effective iterative methods. Unfortunately, fewer elegant results have been discovered since the 1950s and 1960s. The field has moved in other directions. Methods have gained not only in efficiency but also in robustness and in generality. The traditional techniques which required rather complicated procedures to determine optimal acceleration parameters have yielded to the parameter-free conjugate gradient class of methods. The primary aim of this book is to describe some of the best techniques available today, from both preconditioners and accelerators. One of the aims of the book is to provide a good mix of theory and practice. It also addresses some of the current research issues such as parallel implementations and robust preconditioners. The emphasis is on Krylov subspace methods, currently the most practical and common group of techniques used in applications. Although there is a tutorial chapter that covers the discretization of Partial Differential Equations, the book is not biased toward any specific application area. Instead, the matrices are assumed to be general sparse, possibly irregularly structured. The book has been structured in four distinct parts. The first part, Chapters 1 to 4, presents the basic tools. The second part, Chapters 5 to 8, presents projection methods and Krylov subspace techniques. The third part, Chapters 9 and 10, discusses preconditioning. The fourth part, Chapters 11 to 13, discusses parallel implementations and parallel algorithms. Acknowledgments I am grateful to a number of colleagues who proofread or reviewed different versions of the manuscript. Among them are Randy Bramley (University of Indiana at Bloomingtin), Xiao-Chuan Cai (University of Colorado at Boulder), Tony Chan (University of California at Los Angeles), Jane Cullum (IBM, Yorktown Heights), Alan Edelman (Massachussett Institute of Technology), Paul Fischer (Brown University), David Keyes (Old Dominion University), Beresford Parlett (University of California at Berkeley) and Shang-Hua Teng (University of Minnesota). Their numerous comments, corrections, and encouragements were a highly appreciated contribution. In particular, they helped improve the presentation considerably and prompted the addition of a number of topics missing from earlier versions. This book evolved from several successive improvements of a set of lecture notes for the course “Iterative Methods for Linear Systems” which I taught at the University of Minnesota in the last few years. I apologize to those students who used the earlier error-laden and incomplete manuscripts. Their input and criticism contributed significantly to improving the manuscript. I also wish to thank those students at MIT (with Alan Edelman) and UCLA (with Tony Chan) who used this book in manuscript form and provided helpful feedback. My colleagues at the university of Minnesota, staff and faculty members, have helped in different ways. I wish to thank in particular
PREFACE xxi Ahmed Sameh for his encouragements and for fostering a productive environment in the department. Finally, I am grateful to the National Science Foundation for their continued financial support of my research, part of which is represented in this work. Yousef Saad
Page 1: Iterative Methods for Sparse Linear
Page 4 and 5: vi CONTENTS 2.1.2 The Convection Di
Page 6 and 7: viii CONTENTS 6.11.4 Convergence of
Page 8 and 9: x CONTENTS 11.5 Matrix-by-Vector Pr
Page 10 and 11: xii CONTENTS
Page 12 and 13: xiv PREFACE underlying architecture
Page 14 and 15: xvi PREFACE The rest could be skipp
Page 16 and 17: xviii PREFACE
Page 21 and 22: Chapter 1 BACKGROUND IN LINEAR ALGE
Page 23 and 24: 1.2. SQUARE MATRICES AND EIGENVALUE
Page 25 and 26: 1.3. TYPES OF MATRICES 5 • Unitar
Page 27 and 28: 1.4. VECTOR INNER PRODUCTS AND NORM
Page 29 and 30: 1.5. MATRIX NORMS 9 (1.8-1.10) and
Page 31 and 32: 1.7. ORTHOGONAL VECTORS AND SUBSPAC
Page 33 and 34: 1.7. ORTHOGONAL VECTORS AND SUBSPAC
Page 35 and 36: 1.8. CANONICAL FORMS OF MATRICES 15
Page 41 and 42: 1.9. NORMAL AND HERMITIAN MATRICES
Page 47 and 48: 1.10. NONNEGATIVE MATRICES, M-MATRI
Page 53 and 54: 1.11. POSITIVE-DEFINITE MATRICES 33
Page 55 and 56: 1.12. PROJECTION OPERATORS 35 In ad
Page 57 and 58: 1.12. PROJECTION OPERATORS 37 In ma
Page 59 and 60: 1.13. BASIC CONCEPTS IN LINEAR SYST
Page 67 and 68: Chapter 2 DISCRETIZATION OF PDES Pa
Page 69 and 70:
2.1. PARTIAL DIFFERENTIAL EQUATIONS
Page 71 and 72:
2.2. FINITE DIFFERENCE METHODS 51 f
Page 73 and 74:
2.2. FINITE DIFFERENCE METHODS 53 u
Page 75 and 76:
2.2. FINITE DIFFERENCE METHODS 55 B
Page 77 and 78:
2.2. FINITE DIFFERENCE METHODS 57
Page 79 and 80:
2.2. FINITE DIFFERENCE METHODS 59 i
Page 81 and 82:
2.2. FINITE DIFFERENCE METHODS 61 T
Page 83 and 84:
2.3. THE FINITE ELEMENT METHOD 63 o
Page 85 and 86:
2.3. THE FINITE ELEMENT METHOD 65 F
Page 87 and 88:
2.3. THE FINITE ELEMENT METHOD 67 i
Page 89 and 90:
2.4. MESH GENERATION AND REFINEMENT
Page 91 and 92:
2.5. FINITE VOLUME METHOD 71 3 (a)
Page 93 and 94:
2.5. FINITE VOLUME METHOD 73 This g
Page 95 and 96:
Chapter 3 SPARSE MATRICES As descri
Page 97 and 98:
3.2. GRAPH REPRESENTATIONS 77 Figur
Page 99 and 100:
3.3. PERMUTATIONS AND REORDERINGS 7
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
3.4. STORAGE SCHEMES 93 containing
Page 115 and 116:
3.5. BASIC SPARSE MATRIX OPERATIONS
Page 117 and 118:
3.6. SPARSE DIRECT SOLUTION METHODS
Page 119 and 120:
3.7. TEST PROBLEMS 99 3 4 1 4 ✻ a
Page 121 and 122:
3.7. TEST PROBLEMS 101 Figure 3.13:
Page 123 and 124:
3.7. TEST PROBLEMS 103 P-3.8 Consid
Page 125 and 126:
Chapter 4 BASIC ITERATIVE METHODS T
Page 127 and 128:
4.1. JACOBI, GAUSS-SEIDEL, AND SOR
Page 129 and 130:
Page 131 and 132:
Page 133 and 134:
Page 135 and 136:
4.2. CONVERGENCE 115 4.2.1 General
Page 137 and 138:
4.2. CONVERGENCE 117 Then, the eige
Page 139 and 140:
4.2. CONVERGENCE 119 From this and
Page 141 and 142:
4.2. CONVERGENCE 121 cannot be an i
Page 143 and 144:
4.2. CONVERGENCE 123 4.2.5 Property
Page 145 and 146:
4.2. CONVERGENCE 125 Consider now a
Page 147 and 148:
4.3. ALTERNATING DIRECTION METHODS
Page 149 and 150:
Page 151 and 152:
Page 153 and 154:
Chapter 5 PROJECTION METHODS Most o
Page 155 and 156:
5.1. BASIC DEFINITIONS AND ALGORITH
Page 157 and 158:
5.2. GENERAL THEORY 137 5.2 General
Page 159 and 160:
5.2. GENERAL THEORY 139 ✻✯ ❥
Page 161 and 162:
5.2. GENERAL THEORY 141 Thus, an n-
Page 163 and 164:
5.3. ONE-DIMENSIONAL PROJECTION PRO
Page 165 and 166:
5.3. ONE-DIMENSIONAL PROJECTION PRO
Page 167 and 168:
5.4. ADDITIVE AND MULTIPLICATIVE PR
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Chapter 6 KRYLOV SUBSPACE METHODS P
Page 179 and 180:
6.2. KRYLOV SUBSPACES 159 Proof. Th
Page 181 and 182:
6.3. ARNOLDI’S METHOD 161 which s
Page 183 and 184:
6.3. ARNOLDI’S METHOD 163 from a
Page 185 and 186:
6.4. ARNOLDI’S METHOD FOR LINEAR
Page 187 and 188:
Page 189 and 190:
Page 191 and 192:
6.5. GMRES 171 6. Fortunately, an i
Page 193 and 194:
6.5. GMRES 173 6.5.2 The Householde
Page 195 and 196:
6.5. GMRES 175 Define the rotation
Page 197 and 198:
6.5. GMRES 177 3. The residual vect
Page 199 and 200:
6.5. GMRES 179 6.5.4 Breakdown of G
Page 201 and 202:
6.5. GMRES 181 note that if ¯ Hm i
Page 203 and 204:
6.5. GMRES 183 then b − Axm2 = |
Page 205 and 206:
6.5. GMRES 185 Matrix Iters Kflops
Page 207 and 208:
6.5. GMRES 187 Proposition 6.12 Ass
Page 209 and 210:
6.5. GMRES 189 Proof. The following
Page 211 and 212:
6.5. GMRES 191 In the situation whe
Page 213 and 214:
6.5. GMRES 193 QMRES/DQGMRES. The k
Page 215 and 216:
6.6. THE SYMMETRIC LANCZOS ALGORITH
Page 217 and 218:
6.7. THE CONJUGATE GRADIENT ALGORIT
Page 219 and 220:
Page 221 and 222:
Page 223 and 224:
6.8. THE CONJUGATE RESIDUAL METHOD
Page 225 and 226:
6.9. GCR, ORTHOMIN, AND ORTHODIR 20
Page 227 and 228:
6.10. THE FABER-MANTEUFFEL THEOREM
Page 229 and 230:
6.11. CONVERGENCE ANALYSIS 209 One
Page 231 and 232:
6.11. CONVERGENCE ANALYSIS 211 Ther
Page 233 and 234:
6.11. CONVERGENCE ANALYSIS 213 over
Page 235 and 236:
6.11. CONVERGENCE ANALYSIS 215 Ther
Page 237 and 238:
ℑm(z) 6.11. CONVERGENCE ANALYSIS
Page 239 and 240:
6.12. BLOCK KRYLOV METHODS 219 Kryl
Page 241 and 242:
6.12. BLOCK KRYLOV METHODS 221 Now
Page 243 and 244:
6.12. BLOCK KRYLOV METHODS 223 wher
Page 245 and 246:
6.12. BLOCK KRYLOV METHODS 225 P-6.
Page 247 and 248:
6.12. BLOCK KRYLOV METHODS 227 is p
Page 249 and 250:
Chapter 7 KRYLOV SUBSPACE METHODS P
Page 251 and 252:
7.1. LANCZOS BIORTHOGONALIZATION 23
Page 253 and 254:
7.1. LANCZOS BIORTHOGONALIZATION 23
Page 255 and 256:
7.3. THE BCG AND QMR ALGORITHMS 235
Page 257 and 258:
Page 259 and 260:
Page 261 and 262:
7.4. TRANSPOSE-FREE VARIANTS 241 AL
Page 263 and 264:
7.4. TRANSPOSE-FREE VARIANTS 243 th
Page 265 and 266:
7.4. TRANSPOSE-FREE VARIANTS 245 Ac
Page 267 and 268:
7.4. TRANSPOSE-FREE VARIANTS 247 4.
Page 269 and 270:
7.4. TRANSPOSE-FREE VARIANTS 249 Ne
Page 271 and 272:
7.4. TRANSPOSE-FREE VARIANTS 251 Th
Page 273 and 274:
7.4. TRANSPOSE-FREE VARIANTS 253 15
Page 275 and 276:
7.4. TRANSPOSE-FREE VARIANTS 255 P-
Page 277 and 278:
7.4. TRANSPOSE-FREE VARIANTS 257 A
Page 279 and 280:
Chapter 8 METHODS RELATED TO THE NO
Page 281 and 282:
8.2. ROW PROJECTION METHODS 261 the
Page 283 and 284:
8.2. ROW PROJECTION METHODS 263 ALG
Page 285 and 286:
8.2. ROW PROJECTION METHODS 265 in
Page 287 and 288:
8.3. CONJUGATE GRADIENT AND NORMAL
Page 289 and 290:
8.4. SADDLE-POINT PROBLEMS 269 with
Page 291 and 292:
8.4. SADDLE-POINT PROBLEMS 271 Apar
Page 293 and 294:
8.4. SADDLE-POINT PROBLEMS 273 a. S
Page 295 and 296:
Chapter 9 PRECONDITIONED ITERATIONS
Page 297 and 298:
9.2. PRECONDITIONED CONJUGATE GRADI
Page 299 and 300:
Page 301 and 302:
Page 303 and 304:
9.3. PRECONDITIONED GMRES 283 Somet
Page 305 and 306:
9.3. PRECONDITIONED GMRES 285 9.3.3
Page 307 and 308:
9.4. FLEXIBLE VARIANTS 287 In most
Page 309 and 310:
9.4. FLEXIBLE VARIANTS 289 Proposit
Page 311 and 312:
9.5. PRECONDITIONED CG FOR THE NORM
Page 313 and 314:
9.6. THE CONCUS, GOLUB, AND WIDLUND
Page 315 and 316:
9.6. THE CONCUS, GOLUB, AND WIDLUND
Page 317 and 318:
Chapter 10 PRECONDITIONING TECHNIQU
Page 319 and 320:
10.2. JACOBI, SOR, AND SSOR PRECOND
Page 321 and 322:
10.3. ILU FACTORIZATION PRECONDITIO
Page 323 and 324:
Page 325 and 326:
Page 327 and 328:
Page 329 and 330:
Page 331 and 332:
Page 333 and 334:
Page 335 and 336:
Page 337 and 338:
Page 339 and 340:
Page 341 and 342:
10.4. THRESHOLD STRATEGIES AND ILUT
Page 343 and 344:
Page 345 and 346:
Page 347 and 348:
Page 349 and 350:
Page 351 and 352:
Page 353 and 354:
Page 355 and 356:
Page 357 and 358:
10.5. APPROXIMATE INVERSE PRECONDIT
Page 359 and 360:
Page 361 and 362:
Page 363 and 364:
Page 365 and 366:
Page 367 and 368:
Page 369 and 370:
Page 371 and 372:
10.6. REORDERING FOR ILU 351 become
Page 373 and 374:
10.6. REORDERING FOR ILU 353 Rows 1
Page 375 and 376:
10.7. BLOCK PRECONDITIONERS 355 A b
Page 377 and 378:
10.8. PRECONDITIONERS FOR THE NORMA
Page 379 and 380:
Page 381 and 382:
Page 383 and 384:
Page 385 and 386:
Page 387 and 388:
Page 389 and 390:
Chapter 11 PARALLEL IMPLEMENTATIONS
Page 391 and 392:
11.3. TYPES OF PARALLEL ARCHITECTUR
Page 393 and 394:
Page 395 and 396:
Page 397 and 398:
11.4. TYPES OF OPERATIONS 377 among
Page 399 and 400:
11.5. MATRIX-BY-VECTOR PRODUCTS 379
Page 401 and 402:
Page 403 and 404:
Page 405 and 406:
Page 407 and 408:
11.6. STANDARD PRECONDITIONING OPER
Page 409 and 410:
Page 411 and 412:
Page 413 and 414:
Chapter 12 PARALLEL PRECONDITIONERS
Page 415 and 416:
12.3. POLYNOMIAL PRECONDITIONERS 39
Page 417 and 418:
Page 419 and 420:
Page 421 and 422:
Page 423 and 424:
Page 425 and 426:
Page 427 and 428:
12.4. MULTICOLORING 407 22 10 23 11
Page 429 and 430:
12.5. MULTI-ELIMINATION ILU 409 D1
Page 431 and 432:
12.5. MULTI-ELIMINATION ILU 411 as
Page 433 and 434:
12.5. MULTI-ELIMINATION ILU 413 1.
Page 435 and 436:
12.6. DISTRIBUTED ILU AND SSOR 415
Page 437 and 438:
12.7. OTHER TECHNIQUES 417 As in th
Page 439 and 440:
12.7. OTHER TECHNIQUES 419 The prec
Page 441 and 442:
12.7. OTHER TECHNIQUES 421 P-12.3 S
Page 443 and 444:
Chapter 13 MULTIGRID METHODS The co
Page 445 and 446:
13.2. MATRICES AND SPECTRA OF MODEL
Page 447 and 448:
Page 449 and 450:
Page 451 and 452:
Page 453 and 454:
Page 455 and 456:
Page 457 and 458:
13.3. INTER-GRID OPERATIONS 437 and
Page 459 and 460:
13.4. STANDARD MULTIGRID TECHNIQUES
Page 461 and 462:
Page 463 and 464:
Page 465 and 466:
Page 467 and 468:
Page 469 and 470:
Page 471 and 472:
13.5. ANALYSIS FOR THE TWO-GRID CYC
Page 473 and 474:
13.5. ANALYSIS FOR THE TWO-GRID CYC
Page 475 and 476:
13.6. ALGEBRAIC MULTIGRID 455 to sa
Page 477 and 478:
13.6. ALGEBRAIC MULTIGRID 457 It is
Page 479 and 480:
13.6. ALGEBRAIC MULTIGRID 459 all k
Page 481 and 482:
13.6. ALGEBRAIC MULTIGRID 461 • T
Page 483 and 484:
13.6. ALGEBRAIC MULTIGRID 463 1. f
Page 485 and 486:
13.7. MULTIGRID VS KRYLOV METHODS 4
Page 487 and 488:
13.7. MULTIGRID VS KRYLOV METHODS 4
Page 489 and 490:
Chapter 14 DOMAIN DECOMPOSITION MET
Page 491 and 492:
14.1. INTRODUCTION 471 in Figure 14
Page 493 and 494:
14.1. INTRODUCTION 473 4. Subdomain
Page 495 and 496:
14.2. DIRECT SOLUTION AND THE SCHUR
Page 497 and 498:
Page 499 and 500:
Page 501 and 502:
Page 503 and 504:
Page 505 and 506:
14.3. SCHWARZ ALTERNATING PROCEDURE
Page 507 and 508:
Page 509 and 510:
Page 511 and 512:
Page 513 and 514:
Page 515 and 516:
Page 517 and 518:
14.4. SCHUR COMPLEMENT APPROACHES 4
Page 519 and 520:
14.4. SCHUR COMPLEMENT APPROACHES 4
Page 521 and 522:
14.5. FULL MATRIX METHODS 501 inter
Page 523 and 524:
14.5. FULL MATRIX METHODS 503 Assum
Page 525 and 526:
14.6. GRAPH PARTITIONING 505 14.6.2
Page 527 and 528:
14.6. GRAPH PARTITIONING 507 The ma
Page 529 and 530:
14.6. GRAPH PARTITIONING 509 ALGORI
Page 531 and 532:
14.6. GRAPH PARTITIONING 511 Figure
Page 533:
14.6. GRAPH PARTITIONING 513 foo wi
Page 536 and 537:
516 BIBLIOGRAPHY [16] O. AXELSSON,
Page 538 and 539:
518 BIBLIOGRAPHY [48] , An iterativ
Page 540 and 541:
520 BIBLIOGRAPHY [83] N. CHRISOCHOI
Page 542 and 543:
522 BIBLIOGRAPHY [117] H. C. ELMAN
Page 544 and 545:
524 BIBLIOGRAPHY [153] G. H. GOLUB
Page 546 and 547:
526 BIBLIOGRAPHY [188] R. KETTLER,
Page 548 and 549:
528 BIBLIOGRAPHY [222] C. C. PAIGE,
Page 550 and 551:
530 BIBLIOGRAPHY [255] Y. SAAD AND
Page 552 and 553:
532 BIBLIOGRAPHY [289] H. A. VAN DE
Page 554 and 555:
534 BIBLIOGRAPHY [324] L. ZHOU AND
Page 556 and 557:
536 INDEX block GMRES, 222-223 mult
Page 558 and 559:
538 INDEX Element-By-Element precon
Page 560 and 561:
540 INDEX ILU, 301-332 Crout varian
Page 562 and 563:
542 INDEX normal, 4, 21 orthogonal,
Page 564 and 565:
544 INDEX by ILQ, 359 EBE, 417 inco
Page 566 and 567:
546 INDEX permutation and reorderin
show all

Iterative Methods for Sparse Linear Systems Second Edition

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?