Iterative Methods for Sparse Linear Systems Second Edition

More documents

Recommendations

Info

vi CONTENTS 2.1.2 The Convection Diffusion Equation . . . . . . . . 50 2.2 Finite Difference Methods . . . . . . . . . . . . . . . . . . . . 50 2.2.1 Basic Approximations . . . . . . . . . . . . . . . 50 2.2.2 Difference Schemes for the Laplacean Operator . 52 2.2.3 Finite Differences for 1-D Problems . . . . . . . . 53 2.2.4 Upwind Schemes . . . . . . . . . . . . . . . . . 54 2.2.5 Finite Differences for 2-D Problems . . . . . . . . 56 2.2.6 Fast Poisson Solvers . . . . . . . . . . . . . . . . 58 2.3 The Finite Element Method . . . . . . . . . . . . . . . . . . . 62 2.4 Mesh Generation and Refinement . . . . . . . . . . . . . . . . 69 2.5 Finite Volume Method . . . . . . . . . . . . . . . . . . . . . . 69 3 Sparse Matrices 75 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.2 Graph Representations . . . . . . . . . . . . . . . . . . . . . . 76 3.2.1 Graphs and Adjacency Graphs . . . . . . . . . . . 77 3.2.2 Graphs of PDE Matrices . . . . . . . . . . . . . . 78 3.3 Permutations and Reorderings . . . . . . . . . . . . . . . . . . 79 3.3.1 Basic Concepts . . . . . . . . . . . . . . . . . . . 79 3.3.2 Relations with the Adjacency Graph . . . . . . . 81 3.3.3 Common Reorderings . . . . . . . . . . . . . . . 83 3.3.4 Irreducibility . . . . . . . . . . . . . . . . . . . . 91 3.4 Storage Schemes . . . . . . . . . . . . . . . . . . . . . . . . . 92 3.5 Basic Sparse Matrix Operations . . . . . . . . . . . . . . . . . 95 3.6 Sparse Direct Solution Methods . . . . . . . . . . . . . . . . . 96 3.6.1 Minimum degree ordering . . . . . . . . . . . . . 97 3.6.2 Nested Dissection ordering . . . . . . . . . . . . 97 3.7 Test Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 98 4 Basic Iterative Methods 105 4.1 Jacobi, Gauss-Seidel, and SOR . . . . . . . . . . . . . . . . . 105 4.1.1 Block Relaxation Schemes . . . . . . . . . . . . 108 4.1.2 Iteration Matrices and Preconditioning . . . . . . 112 4.2 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 4.2.1 General Convergence Result . . . . . . . . . . . . 115 4.2.2 Regular Splittings . . . . . . . . . . . . . . . . . 118 4.2.3 Diagonally Dominant Matrices . . . . . . . . . . 119 4.2.4 Symmetric Positive Definite Matrices . . . . . . . 122 4.2.5 Property A and Consistent Orderings . . . . . . . 123 4.3 Alternating Direction Methods . . . . . . . . . . . . . . . . . 127 5 Projection Methods 133 5.1 Basic Definitions and Algorithms . . . . . . . . . . . . . . . . 133 5.1.1 General Projection Methods . . . . . . . . . . . . 134
CONTENTS vii 5.1.2 Matrix Representation . . . . . . . . . . . . . . . 135 5.2 General Theory . . . . . . . . . . . . . . . . . . . . . . . . . 137 5.2.1 Two Optimality Results . . . . . . . . . . . . . . 137 5.2.2 Interpretation in Terms of Projectors . . . . . . . 138 5.2.3 General Error Bound . . . . . . . . . . . . . . . . 140 5.3 One-Dimensional Projection Processes . . . . . . . . . . . . . 142 5.3.1 Steepest Descent . . . . . . . . . . . . . . . . . . 142 5.3.2 Minimal Residual (MR) Iteration . . . . . . . . . 145 5.3.3 Residual Norm Steepest Descent . . . . . . . . . 147 5.4 Additive and Multiplicative Processes . . . . . . . . . . . . . . 147 6 Krylov Subspace Methods Part I 157 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 6.2 Krylov Subspaces . . . . . . . . . . . . . . . . . . . . . . . . 158 6.3 Arnoldi’s Method . . . . . . . . . . . . . . . . . . . . . . . . 160 6.3.1 The Basic Algorithm . . . . . . . . . . . . . . . . 160 6.3.2 Practical Implementations . . . . . . . . . . . . . 162 6.4 Arnoldi’s Method for Linear Systems (FOM) . . . . . . . . . . 165 6.4.1 Variation 1: Restarted FOM . . . . . . . . . . . . 167 6.4.2 Variation 2: IOM and DIOM . . . . . . . . . . . 168 6.5 GMRES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 6.5.1 The Basic GMRES Algorithm . . . . . . . . . . . 172 6.5.2 The Householder Version . . . . . . . . . . . . . 173 6.5.3 Practical Implementation Issues . . . . . . . . . . 174 6.5.4 Breakdown of GMRES . . . . . . . . . . . . . . 179 6.5.5 Variation 1: Restarting . . . . . . . . . . . . . . . 179 6.5.6 Variation 2: Truncated GMRES Versions . . . . . 180 6.5.7 Relations between FOM and GMRES . . . . . . . 185 6.5.8 Residual smoothing . . . . . . . . . . . . . . . . 190 6.5.9 GMRES for complex systems . . . . . . . . . . . 193 6.6 The Symmetric Lanczos Algorithm . . . . . . . . . . . . . . . 194 6.6.1 The Algorithm . . . . . . . . . . . . . . . . . . . 194 6.6.2 Relation with Orthogonal Polynomials . . . . . . 195 6.7 The Conjugate Gradient Algorithm . . . . . . . . . . . . . . . 196 6.7.1 Derivation and Theory . . . . . . . . . . . . . . . 196 6.7.2 Alternative Formulations . . . . . . . . . . . . . 200 6.7.3 Eigenvalue Estimates from the CG Coefficients . . 201 6.8 The Conjugate Residual Method . . . . . . . . . . . . . . . . 203 6.9 GCR, ORTHOMIN, and ORTHODIR . . . . . . . . . . . . . . 204 6.10 The Faber-Manteuffel Theorem . . . . . . . . . . . . . . . . . 206 6.11 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . 208 6.11.1 Real Chebyshev Polynomials . . . . . . . . . . . 209 6.11.2 Complex Chebyshev Polynomials . . . . . . . . . 210 6.11.3 Convergence of the CG Algorithm . . . . . . . . 214
Page 1: Iterative Methods for Sparse Linear
Page 6 and 7: viii CONTENTS 6.11.4 Convergence of
Page 8 and 9: x CONTENTS 11.5 Matrix-by-Vector Pr
Page 10 and 11: xii CONTENTS
Page 12 and 13: xiv PREFACE underlying architecture
Page 14 and 15: xvi PREFACE The rest could be skipp
Page 16 and 17: xviii PREFACE
Page 18 and 19: xx PREFACE Apart from two recent vo
Page 21 and 22: Chapter 1 BACKGROUND IN LINEAR ALGE
Page 23 and 24: 1.2. SQUARE MATRICES AND EIGENVALUE
Page 25 and 26: 1.3. TYPES OF MATRICES 5 • Unitar
Page 27 and 28: 1.4. VECTOR INNER PRODUCTS AND NORM
Page 29 and 30: 1.5. MATRIX NORMS 9 (1.8-1.10) and
Page 31 and 32: 1.7. ORTHOGONAL VECTORS AND SUBSPAC
Page 33 and 34: 1.7. ORTHOGONAL VECTORS AND SUBSPAC
Page 35 and 36: 1.8. CANONICAL FORMS OF MATRICES 15
Page 41 and 42: 1.9. NORMAL AND HERMITIAN MATRICES
Page 47 and 48: 1.10. NONNEGATIVE MATRICES, M-MATRI
Page 53 and 54: 1.11. POSITIVE-DEFINITE MATRICES 33
Page 55 and 56:
1.12. PROJECTION OPERATORS 35 In ad
Page 57 and 58:
1.12. PROJECTION OPERATORS 37 In ma
Page 59 and 60:
1.13. BASIC CONCEPTS IN LINEAR SYST
Page 61 and 62:
Page 63 and 64:
Page 65 and 66:
Page 67 and 68:
Chapter 2 DISCRETIZATION OF PDES Pa
Page 69 and 70:
2.1. PARTIAL DIFFERENTIAL EQUATIONS
Page 71 and 72:
2.2. FINITE DIFFERENCE METHODS 51 f
Page 73 and 74:
2.2. FINITE DIFFERENCE METHODS 53 u
Page 75 and 76:
2.2. FINITE DIFFERENCE METHODS 55 B
Page 77 and 78:
2.2. FINITE DIFFERENCE METHODS 57
Page 79 and 80:
2.2. FINITE DIFFERENCE METHODS 59 i
Page 81 and 82:
2.2. FINITE DIFFERENCE METHODS 61 T
Page 83 and 84:
2.3. THE FINITE ELEMENT METHOD 63 o
Page 85 and 86:
2.3. THE FINITE ELEMENT METHOD 65 F
Page 87 and 88:
2.3. THE FINITE ELEMENT METHOD 67 i
Page 89 and 90:
2.4. MESH GENERATION AND REFINEMENT
Page 91 and 92:
2.5. FINITE VOLUME METHOD 71 3 (a)
Page 93 and 94:
2.5. FINITE VOLUME METHOD 73 This g
Page 95 and 96:
Chapter 3 SPARSE MATRICES As descri
Page 97 and 98:
3.2. GRAPH REPRESENTATIONS 77 Figur
Page 99 and 100:
3.3. PERMUTATIONS AND REORDERINGS 7
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
3.4. STORAGE SCHEMES 93 containing
Page 115 and 116:
3.5. BASIC SPARSE MATRIX OPERATIONS
Page 117 and 118:
3.6. SPARSE DIRECT SOLUTION METHODS
Page 119 and 120:
3.7. TEST PROBLEMS 99 3 4 1 4 ✻ a
Page 121 and 122:
3.7. TEST PROBLEMS 101 Figure 3.13:
Page 123 and 124:
3.7. TEST PROBLEMS 103 P-3.8 Consid
Page 125 and 126:
Chapter 4 BASIC ITERATIVE METHODS T
Page 127 and 128:
4.1. JACOBI, GAUSS-SEIDEL, AND SOR
Page 129 and 130:
Page 131 and 132:
Page 133 and 134:
Page 135 and 136:
4.2. CONVERGENCE 115 4.2.1 General
Page 137 and 138:
4.2. CONVERGENCE 117 Then, the eige
Page 139 and 140:
4.2. CONVERGENCE 119 From this and
Page 141 and 142:
4.2. CONVERGENCE 121 cannot be an i
Page 143 and 144:
4.2. CONVERGENCE 123 4.2.5 Property
Page 145 and 146:
4.2. CONVERGENCE 125 Consider now a
Page 147 and 148:
4.3. ALTERNATING DIRECTION METHODS
Page 149 and 150:
Page 151 and 152:
Page 153 and 154:
Chapter 5 PROJECTION METHODS Most o
Page 155 and 156:
5.1. BASIC DEFINITIONS AND ALGORITH
Page 157 and 158:
5.2. GENERAL THEORY 137 5.2 General
Page 159 and 160:
5.2. GENERAL THEORY 139 ✻✯ ❥
Page 161 and 162:
5.2. GENERAL THEORY 141 Thus, an n-
Page 163 and 164:
5.3. ONE-DIMENSIONAL PROJECTION PRO
Page 165 and 166:
5.3. ONE-DIMENSIONAL PROJECTION PRO
Page 167 and 168:
5.4. ADDITIVE AND MULTIPLICATIVE PR
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Chapter 6 KRYLOV SUBSPACE METHODS P
Page 179 and 180:
6.2. KRYLOV SUBSPACES 159 Proof. Th
Page 181 and 182:
6.3. ARNOLDI’S METHOD 161 which s
Page 183 and 184:
6.3. ARNOLDI’S METHOD 163 from a
Page 185 and 186:
6.4. ARNOLDI’S METHOD FOR LINEAR
Page 187 and 188:
Page 189 and 190:
Page 191 and 192:
6.5. GMRES 171 6. Fortunately, an i
Page 193 and 194:
6.5. GMRES 173 6.5.2 The Householde
Page 195 and 196:
6.5. GMRES 175 Define the rotation
Page 197 and 198:
6.5. GMRES 177 3. The residual vect
Page 199 and 200:
6.5. GMRES 179 6.5.4 Breakdown of G
Page 201 and 202:
6.5. GMRES 181 note that if ¯ Hm i
Page 203 and 204:
6.5. GMRES 183 then b − Axm2 = |
Page 205 and 206:
6.5. GMRES 185 Matrix Iters Kflops
Page 207 and 208:
6.5. GMRES 187 Proposition 6.12 Ass
Page 209 and 210:
6.5. GMRES 189 Proof. The following
Page 211 and 212:
6.5. GMRES 191 In the situation whe
Page 213 and 214:
6.5. GMRES 193 QMRES/DQGMRES. The k
Page 215 and 216:
6.6. THE SYMMETRIC LANCZOS ALGORITH
Page 217 and 218:
6.7. THE CONJUGATE GRADIENT ALGORIT
Page 219 and 220:
Page 221 and 222:
Page 223 and 224:
6.8. THE CONJUGATE RESIDUAL METHOD
Page 225 and 226:
6.9. GCR, ORTHOMIN, AND ORTHODIR 20
Page 227 and 228:
6.10. THE FABER-MANTEUFFEL THEOREM
Page 229 and 230:
6.11. CONVERGENCE ANALYSIS 209 One
Page 231 and 232:
6.11. CONVERGENCE ANALYSIS 211 Ther
Page 233 and 234:
6.11. CONVERGENCE ANALYSIS 213 over
Page 235 and 236:
6.11. CONVERGENCE ANALYSIS 215 Ther
Page 237 and 238:
ℑm(z) 6.11. CONVERGENCE ANALYSIS
Page 239 and 240:
6.12. BLOCK KRYLOV METHODS 219 Kryl
Page 241 and 242:
6.12. BLOCK KRYLOV METHODS 221 Now
Page 243 and 244:
6.12. BLOCK KRYLOV METHODS 223 wher
Page 245 and 246:
6.12. BLOCK KRYLOV METHODS 225 P-6.
Page 247 and 248:
6.12. BLOCK KRYLOV METHODS 227 is p
Page 249 and 250:
Chapter 7 KRYLOV SUBSPACE METHODS P
Page 251 and 252:
7.1. LANCZOS BIORTHOGONALIZATION 23
Page 253 and 254:
7.1. LANCZOS BIORTHOGONALIZATION 23
Page 255 and 256:
7.3. THE BCG AND QMR ALGORITHMS 235
Page 257 and 258:
Page 259 and 260:
Page 261 and 262:
7.4. TRANSPOSE-FREE VARIANTS 241 AL
Page 263 and 264:
7.4. TRANSPOSE-FREE VARIANTS 243 th
Page 265 and 266:
7.4. TRANSPOSE-FREE VARIANTS 245 Ac
Page 267 and 268:
7.4. TRANSPOSE-FREE VARIANTS 247 4.
Page 269 and 270:
7.4. TRANSPOSE-FREE VARIANTS 249 Ne
Page 271 and 272:
7.4. TRANSPOSE-FREE VARIANTS 251 Th
Page 273 and 274:
7.4. TRANSPOSE-FREE VARIANTS 253 15
Page 275 and 276:
7.4. TRANSPOSE-FREE VARIANTS 255 P-
Page 277 and 278:
7.4. TRANSPOSE-FREE VARIANTS 257 A
Page 279 and 280:
Chapter 8 METHODS RELATED TO THE NO
Page 281 and 282:
8.2. ROW PROJECTION METHODS 261 the
Page 283 and 284:
8.2. ROW PROJECTION METHODS 263 ALG
Page 285 and 286:
8.2. ROW PROJECTION METHODS 265 in
Page 287 and 288:
8.3. CONJUGATE GRADIENT AND NORMAL
Page 289 and 290:
8.4. SADDLE-POINT PROBLEMS 269 with
Page 291 and 292:
8.4. SADDLE-POINT PROBLEMS 271 Apar
Page 293 and 294:
8.4. SADDLE-POINT PROBLEMS 273 a. S
Page 295 and 296:
Chapter 9 PRECONDITIONED ITERATIONS
Page 297 and 298:
9.2. PRECONDITIONED CONJUGATE GRADI
Page 299 and 300:
Page 301 and 302:
Page 303 and 304:
9.3. PRECONDITIONED GMRES 283 Somet
Page 305 and 306:
9.3. PRECONDITIONED GMRES 285 9.3.3
Page 307 and 308:
9.4. FLEXIBLE VARIANTS 287 In most
Page 309 and 310:
9.4. FLEXIBLE VARIANTS 289 Proposit
Page 311 and 312:
9.5. PRECONDITIONED CG FOR THE NORM
Page 313 and 314:
9.6. THE CONCUS, GOLUB, AND WIDLUND
Page 315 and 316:
9.6. THE CONCUS, GOLUB, AND WIDLUND
Page 317 and 318:
Chapter 10 PRECONDITIONING TECHNIQU
Page 319 and 320:
10.2. JACOBI, SOR, AND SSOR PRECOND
Page 321 and 322:
10.3. ILU FACTORIZATION PRECONDITIO
Page 323 and 324:
Page 325 and 326:
Page 327 and 328:
Page 329 and 330:
Page 331 and 332:
Page 333 and 334:
Page 335 and 336:
Page 337 and 338:
Page 339 and 340:
Page 341 and 342:
10.4. THRESHOLD STRATEGIES AND ILUT
Page 343 and 344:
Page 345 and 346:
Page 347 and 348:
Page 349 and 350:
Page 351 and 352:
Page 353 and 354:
Page 355 and 356:
Page 357 and 358:
10.5. APPROXIMATE INVERSE PRECONDIT
Page 359 and 360:
Page 361 and 362:
Page 363 and 364:
Page 365 and 366:
Page 367 and 368:
Page 369 and 370:
Page 371 and 372:
10.6. REORDERING FOR ILU 351 become
Page 373 and 374:
10.6. REORDERING FOR ILU 353 Rows 1
Page 375 and 376:
10.7. BLOCK PRECONDITIONERS 355 A b
Page 377 and 378:
10.8. PRECONDITIONERS FOR THE NORMA
Page 379 and 380:
Page 381 and 382:
Page 383 and 384:
Page 385 and 386:
Page 387 and 388:
Page 389 and 390:
Chapter 11 PARALLEL IMPLEMENTATIONS
Page 391 and 392:
11.3. TYPES OF PARALLEL ARCHITECTUR
Page 393 and 394:
Page 395 and 396:
Page 397 and 398:
11.4. TYPES OF OPERATIONS 377 among
Page 399 and 400:
11.5. MATRIX-BY-VECTOR PRODUCTS 379
Page 401 and 402:
Page 403 and 404:
Page 405 and 406:
Page 407 and 408:
11.6. STANDARD PRECONDITIONING OPER
Page 409 and 410:
Page 411 and 412:
Page 413 and 414:
Chapter 12 PARALLEL PRECONDITIONERS
Page 415 and 416:
12.3. POLYNOMIAL PRECONDITIONERS 39
Page 417 and 418:
Page 419 and 420:
Page 421 and 422:
Page 423 and 424:
Page 425 and 426:
Page 427 and 428:
12.4. MULTICOLORING 407 22 10 23 11
Page 429 and 430:
12.5. MULTI-ELIMINATION ILU 409 D1
Page 431 and 432:
12.5. MULTI-ELIMINATION ILU 411 as
Page 433 and 434:
12.5. MULTI-ELIMINATION ILU 413 1.
Page 435 and 436:
12.6. DISTRIBUTED ILU AND SSOR 415
Page 437 and 438:
12.7. OTHER TECHNIQUES 417 As in th
Page 439 and 440:
12.7. OTHER TECHNIQUES 419 The prec
Page 441 and 442:
12.7. OTHER TECHNIQUES 421 P-12.3 S
Page 443 and 444:
Chapter 13 MULTIGRID METHODS The co
Page 445 and 446:
13.2. MATRICES AND SPECTRA OF MODEL
Page 447 and 448:
Page 449 and 450:
Page 451 and 452:
Page 453 and 454:
Page 455 and 456:
Page 457 and 458:
13.3. INTER-GRID OPERATIONS 437 and
Page 459 and 460:
13.4. STANDARD MULTIGRID TECHNIQUES
Page 461 and 462:
Page 463 and 464:
Page 465 and 466:
Page 467 and 468:
Page 469 and 470:
Page 471 and 472:
13.5. ANALYSIS FOR THE TWO-GRID CYC
Page 473 and 474:
13.5. ANALYSIS FOR THE TWO-GRID CYC
Page 475 and 476:
13.6. ALGEBRAIC MULTIGRID 455 to sa
Page 477 and 478:
13.6. ALGEBRAIC MULTIGRID 457 It is
Page 479 and 480:
13.6. ALGEBRAIC MULTIGRID 459 all k
Page 481 and 482:
13.6. ALGEBRAIC MULTIGRID 461 • T
Page 483 and 484:
13.6. ALGEBRAIC MULTIGRID 463 1. f
Page 485 and 486:
13.7. MULTIGRID VS KRYLOV METHODS 4
Page 487 and 488:
13.7. MULTIGRID VS KRYLOV METHODS 4
Page 489 and 490:
Chapter 14 DOMAIN DECOMPOSITION MET
Page 491 and 492:
14.1. INTRODUCTION 471 in Figure 14
Page 493 and 494:
14.1. INTRODUCTION 473 4. Subdomain
Page 495 and 496:
14.2. DIRECT SOLUTION AND THE SCHUR
Page 497 and 498:
Page 499 and 500:
Page 501 and 502:
Page 503 and 504:
Page 505 and 506:
14.3. SCHWARZ ALTERNATING PROCEDURE
Page 507 and 508:
Page 509 and 510:
Page 511 and 512:
Page 513 and 514:
Page 515 and 516:
Page 517 and 518:
14.4. SCHUR COMPLEMENT APPROACHES 4
Page 519 and 520:
14.4. SCHUR COMPLEMENT APPROACHES 4
Page 521 and 522:
14.5. FULL MATRIX METHODS 501 inter
Page 523 and 524:
14.5. FULL MATRIX METHODS 503 Assum
Page 525 and 526:
14.6. GRAPH PARTITIONING 505 14.6.2
Page 527 and 528:
14.6. GRAPH PARTITIONING 507 The ma
Page 529 and 530:
14.6. GRAPH PARTITIONING 509 ALGORI
Page 531 and 532:
14.6. GRAPH PARTITIONING 511 Figure
Page 533:
14.6. GRAPH PARTITIONING 513 foo wi
Page 536 and 537:
516 BIBLIOGRAPHY [16] O. AXELSSON,
Page 538 and 539:
518 BIBLIOGRAPHY [48] , An iterativ
Page 540 and 541:
520 BIBLIOGRAPHY [83] N. CHRISOCHOI
Page 542 and 543:
522 BIBLIOGRAPHY [117] H. C. ELMAN
Page 544 and 545:
524 BIBLIOGRAPHY [153] G. H. GOLUB
Page 546 and 547:
526 BIBLIOGRAPHY [188] R. KETTLER,
Page 548 and 549:
528 BIBLIOGRAPHY [222] C. C. PAIGE,
Page 550 and 551:
530 BIBLIOGRAPHY [255] Y. SAAD AND
Page 552 and 553:
532 BIBLIOGRAPHY [289] H. A. VAN DE
Page 554 and 555:
534 BIBLIOGRAPHY [324] L. ZHOU AND
Page 556 and 557:
536 INDEX block GMRES, 222-223 mult
Page 558 and 559:
538 INDEX Element-By-Element precon
Page 560 and 561:
540 INDEX ILU, 301-332 Crout varian
Page 562 and 563:
542 INDEX normal, 4, 21 orthogonal,
Page 564 and 565:
544 INDEX by ILQ, 359 EBE, 417 inco
Page 566 and 567:
546 INDEX permutation and reorderin
show all

Iterative Methods for Sparse Linear Systems Second Edition

Create successful ePaper yourself

Delete template?

Save as template?