COPYRIGHT 2008, PRINCETON UNIVERSITY PRESS

systems of equations with matrices; data fitting 163As we indicated for the single-equation Newton–Raphson method, while for ourtwo-mass problem we can derive analytic expressions for the derivatives ∂f i /∂x j ,there are 9 × 9=81such derivatives for this (small) problem, and entering themall would be both time-consuming and error-prone. In contrast, especially formore complicated problems, it is straightforward to program a forward-differenceapproximation for the derivatives,∂f i∂x j≃ f i(x j +∆x j ) − f i (x j )δx j, (8.18)where each individual x j is varied independently since these are partial derivativesand δx j are some arbitrary changes you input. While a central-difference approximationfor the derivative would be more accurate, it would also require moreevaluations of the f’s, and once we find a solution it does not matter how accurateour algorithm for the derivative was.As also discussed for the 1-D Newton–Raphson method (§7.10.1), the methodcan fail if the initial guess is not close enough to the zero of f (here all N of them)for the f’s to be approximated as linear. The backtracking technique may be appliedhere as well, in the present case, progressively decreasing the corrections ∆x i until|f| 2 = |f 1 | 2 + |f 2 | 2 + ···+ |f N | 2 decreases.8.3 Classes of Matrix Problems (Math)It helps to remember that the rules of mathematics apply even to the world’smost powerful computers. For example, you should have problems solving equationsif you have more unknowns than equations or if your equations are notlinearly independent. But do not fret. While you cannot obtain a unique solutionwhen there are not enough equations, you may still be able to map out a spaceof allowable solutions. At the other extreme, if you have more equations thanunknowns, you have an overdetermined problem, which may not have a uniquesolution. An overdetermined problem is sometimes treated using data fitting inwhich a solution to a sufficient set of equations is found, tested on the unusedequations, and then improved if needed. Not surprisingly, this latter technique isknown as the linear least-squares method because it finds the best solution “on theaverage.”The most basic matrix problem is the system of linear equations you have tosolve for the two-mass problem:Ax = b, A N×N x N×1 = b N×1 , (8.19)where A is a known N × N matrix, x is an unknown vector of length N, and b isa known vector of length N. The best way to solve this equation is by Gaussianelimination or lower-upper (LU) decomposition. This yields the vector x withoutexplicitly calculating A −1 . Another, albeit slower and less robust, method is todetermine the inverse of A and then form the solution by multiplying both sides−101COPYRIGHT 2008, PRINCET O N UNIVE R S I T Y P R E S SEVALUATION COPY ONLY. NOT FOR USE IN COURSES.ALLpup_06.04 — 2008/2/15 — Page 163