METHODS FOR CALCULATING FRÉCHET DERIVATIVES AND ...

Geophysical Prospecting 38,499-524, 1990METHODS FOR CALCULATING FRÉCHETDERIVATIVES AND SENSITIVITIESFOR THE NON-LINEAR INVERSE PROBLEM:A COMPARATIVE STUDY'P. R. McGILLIVRAY and D. W. OLDENBURG2ABSTRACTMCGILLIVRAY and OLDENBURG, D.W. 1990. Methods for calculating Fréchet derivatives andsensitivities for the non-linear inverse problem: a comparative study. Geophysical Prospecting38,499-524.A fundamental step in the solution of most non-linear inverse problems is to establish arelationship between changes in a proposed model and resulting changes in the forward modelleddata. Once this relationship has been established, it becomes possible to refine an initialmodel to obtain an improved fit to the observed data. In a linearized analysis, the Fréchetderivative is the connecting link between changes in the model and changes in the data. Insome simple cases an analytic expression for the Fréchet derivative may be derived. In thispaper we present three techniques to accomplish this and illustrate them by computing theFréchet derivative for the 1D resistivity problem. For more complicated problems, where it isnot possible to obtain an expression for the Fréchet derivative, it is necessary to parameterizethe model and solve numerically for the sensitivities - partial derivatives of the data withrespect to model parameters. The standard perturbation method for computing first-ordersensitivities is discussed and compared to the more efficient sensitivity-equation and adjointequationmethods. Extensions to allow for the calculation of higher order, directional andobjective function sensitivities are also presented. Finally, the application of these varioustechniques is illustrated for both the 1D and 2D resistivity problems.INTRODUCTIONLinearized analysis, traditionally employed in solving non-linear inverse problems,demands that either the Fréchet derivative or the first-order sensitivities be computedto evaluate quantitatively how a change in the model affects a datum. Literaturein various fields illustrates how these quantities can be computed; however,there does not exist a detailed comparison of the various approaches which areavailable. As such, when faced with solving an inverse problem, there is often indecisionabout what options exist and how best to find an analytic expression for theReceived April 1989, revision accepted October 1989.Department of Geophysics and Astronomy, University of British Columbia, Vancouver,B.C., Canada V6T 1W5.499

500 P. R. McGILLIVRAY AND D. W. OLDENBURGFréchet derivative or how to compute the sensitivities numerically. The goal of thispaper is to fill that void. Our hope is that at least one of the techniques offered herewill be of use to any practitioner faced with inverting geophysical data.We begin with a general discussion of the forward and inverse problems, whichinvolve mappings between ‘model space’ and ‘data space’. For the work presentedhere, model space is a Hilbert space of functions defined over a suitable interval anddata space is an N-dimensional Euclidean vector space whose elements are real orcomplex numbers.We can represent the forward mapping mathematically byej = Fj(m), j = 1, 2, . . ., N (1)where FXm) is a functional which relates a given model m(x) to the jth datum ej. Ifthe problem is linear then (1) can be expressed asrej = J K,(x)m(x) d3x, j = 1, 2, ..., N,Dwhere KJ(x) is the kernel function associated with the jth datum and D is thedomain of the problem.In many cases the physics of the experiment leads to a description of the forwardproblem in terms of a differential equation and a set of boundary conditions, whichtogether define a mathematical boundary-value problem. The steady-state diffusionproblem, for example, is described by the differential equationLU = -V * (~(x)VU) + ~(x)u = Q(x), (34valid over the domain D, and the boundary conditionMU = ~(x)u+ P(x) - = g(X),auanvalid on the boundary aD of D. In (3), U(X) is the response function to be solved for,p(x) and q(x) jointly specify the model, and Q(x) and g(x) describe the source distribution(or system excitation). Although for simple geometries it may be possibleto find a closed-form solution to (3), generally one is forced to appeal to numericaltechniques.In the inverse problem we seek to find a model m*(x) such thateTbs = FJ(m*), j = 1, 2, . . ., N (4)where eTbs is the jth observation. If the forward problem is linear then a variety ofstandard techniques can be used to solve the set of equations in (4) for m*(x) (e.g.Parker 1977b; Menke 1984; Oldenburg 1984); the particular technique used definesthe inverse mapping. Although non-iterative inverse mappings can be found fornon-linear problems (e.g. Gel’fand-Levitan approaches for inverse scatteringproblems), the techniques are often unstable in the presence of noise. Also, for manynon-linear problems encountered in geophysics, no direct inverse mapping has beenfound. The usual strategy in these cases is to start with an estimated solution,mest(x), and to solve the forward problem to obtain the predicted data. A pertur-(3b)

CALCULATING FRÉCHET DERIVATIVES 501bation dm(x) is sought which, when added to mest(x), yields a model which reproducesthe observed data.To derive equations which accomplish this, we write the jth observation in termsof the expansionwhere the operator Fy)(m) is called the nth order Fréchet derivative of Fj(m) (Griffe11981; Zeidler 1985). The first-order derivative F?)(m) is referred to simply as theFréchet derivative.Letting dej = egbs - Fjm,,,) be the misfit for the jth observation, then (5) can bewritten asdej = FY)(m,,,)dm + 0(116ml12). (6)If the higher-order terms represented by O( 11dmll 2, are neglected, then (6) can bewritten asdej = J>{x, m,,,)dm(x) d3x, (7)where KJ{x, m) is the Fréchet kernel associated with the jth observation. It is thiskernel which establishes the relationship between a small (first-order) perturbationin the model and the corresponding change in the datum. Since (7) is linear, theperturbation dm(x) can be readily computed using standard techniques once ananalytic expression for the Fréchet kernel has been derived.Rather than deriving an expression for the Fréchet derivative, an alternateapproach to the solution of the inverse problem is first to represent the model by afinite set of parameters mk , k = 1, 2, . . . , M. The model is then a Euclidean vector mand the Taylor series expansion for the jth datum iswhere anFJ{m)/amk am, ..., is the nth order sensitivity of Fjm) with respect to thekth, lth, . . . , parameters. Equation (8) can also be written asIf the higher-order terms encompassed by O( 116mll 2, are neglected, one obtains thematrix equationde x Jam, (10)where J is the N x M Jacobian matrix whose elements Jjk = aFj/dmk are the firstordersensitivities. Practical formulations for finding the model perturbation 6mmake use of a generalized inverse for J (Jackson 1972; Wiggins 1972; Jupp andVozoff 1975; Menke 1984).

502 P. R. McGILLIVRAY AND D. W. OLDENBURGThe specific methods used to solve (7) or (10) will differ among inverse practitioners.Also, each problem is b st tackled by introducing an appropriate weightednorm to be minimized and often different regularization schemes are implemented.Nevertheless, almost all linearized inversion methods make use of one of these twoequations and thus the calculation of Fréchet derivative kernels and parametersensitivities is of fundamental importance to the solution of the non-linear inverseproblem.DERIVATION OF ANALYTIC FRÉCHET DERIVATIVESThe usual approach for deriving an expression for the Fréchet derivative is toperturb the governing differential equation and thereby formulate a new problemwhich relates the change in the forward response to a small perturbation in themodel. Let the forward problem be described byLU = Q(x). (1 1)Then the response perturbation Su(x) satisfies, to first order, the problemorLSU + SLU = O, (12)LSU = R(x), (13)where SL is a new operator obtained by perturbing the model, and R(x) = -6Lu.The solution of (13), evaluated at an observation location xo, yields the Fréchetderivative for that observation.Adjoint Green’s function approachA general way of solving (13) for the Fréchet derivative is to make use of anadjoint Green’s function solution (Lanczos 1960; Roach 1982). We begin by definingan adjoint operator L* and considering the problemL*G* = S(X - xO), (14)where G*(x, xo) is the adjoint Green’s function. Multiplying (13) by G* and (14) bySu and subtracting, yields the expressionG*LSu - SuL*G* = G*R(x) - SU(X)S(X - ~ 0). (15)Integrating both sides of (15) over the domain D, and making use of the propertiesof the Dirac delta function, yieldss,S,(G*L6u - SuL*G*) d3x = G*(x, x,)R(x) d3x - Su(xo).(16)If the operator L* and the adjoint boundary conditions are chosen such that(G*LSu- SuL*G*) d3x = O,J,

W),CALCULATING FRÉCHET DERIVATIVES 503for all G* and du, then the Fréchet derivative is given byG*(x, xo)R(x) d3x.Requiring the adjoint problem to satisfy (17) establishes a form of reciprocitybetween the response perturbation and adjoint problems. This makes it possible tocompute the response perturbation du(x,) due to a unit source at x by placing a unitsource at xo and calculating the value of the adjoint Green’s function at x. Equation(18) then uses the linearity of the problem to obtain the response perturbation dueto the complete source term R(x) by superposition. In many cases it is much easierto solve for the adjoint Green’s function and perform the required integrations thanto solve (13) directly.To illustrate the adjoint approach, we derive the Fréchet derivative for the d.c.resistivity problem. The general governing equations arev . (a(x)V4) = O, (1944k Y, Z ) L m = O, ( 1 94where 4(x) is the potential due to a single current electrode located at x, = (O, O, O)and a(x) is the subsurface conductivity distribution. When the conductivity variesonly with depth, (19) can be written asa-az 4(r, O) = - ~I27ca(O)r4@, 4 Lm = O, (20c)where r is the radial distance from the current electrode.Symmetry suggests taking the Hankel transform of 4where Jo(Ar) is a Bessel function of the first kind of order O. The transformedresponse h(A, z) = A&, z) can then be shown to satisfy the forward problem-_ d2hdz2dhW(Z) -- A2h = O,dzdill- h(A7 O) = --dz27ca(0)’

504 P. R. McGILLIVRAY AND D. W. OLDENBURGwhere1 da(z) 1 dp(z)w(z)= ---=--.~(z) dz p(z) dzAs a final transformation it is convenient to define the normalized response$(A, z) = h(Â, z)/h'(Â, O) which satisfiesd2$--dzZdS"dzw(~) - - n2S" = O,d -- S(A, o) = 1,dz$(A, z) La) = o. (234Equations (23a, b, c) are the desired governing differential equation and boundaryconditions for which the Fréchet derivative will be calculated.Since p(z) is expected to vary over several orders of magnitude and is strictlypositive it is convenient to consider either m(z) = In p(z) or w(z) = (d/dz) In p(z) asthe model. In many cases w(z) is the appropriate choice since it will lead to smootherestimates of In p(z). If w(z) is selected as the model, then a perturbed system isobtained by letting w(z) -, w(z) + 6w(z) in (23). Subtracting the unperturbed from theperturbed system yields an equation for the response perturbation 6S"(A, z):d26S" d6S d$-- w(z) -- A26S = 6w(z) -,dz2 dz dzd- dS"((n, O) = O,dz6$((n, Z)Iz-tm = o. (244The adjoint problem which satisfies the reciprocity relationship in (17) can be foundby multiplying (24a) by the adjoint Green's function G*(A, z, 5) and integrating overz. We obtainThe first two terms on the left are then integrated by parts to yieldIm Im G* - 6 $pz dz + wG*) + Srn&fqdz + dz (w(z)G*)- Â2G*] dzdz O= [G*(Â, z, O) - ') 6w(z)dz. (26)dz

CALCULATING FRÉCHET DERIVATIVES 505If we choose the adjoint problem to bed2G*dz2ddz__ + - (w(z)G*) - A2G* = 6(z - 0,d- G*(A, O,

506 P. R. McGILLIVRAY AND D. W. OLDENBURGThe perturbed response satisfies1E2E2[s + &SI + - s2 + * *. - (w + EU)dz 2!dz2 2!s + ES1 + - s2 + . . .s" + &fl11E2+ - S2 + = O. (32)2!Collecting terms involving the same power of E, one obtains the series of equationsd2$k dgk dgk - 1--dz2 dz dzW(Z) - - A'Sk = kq - k = 1, 2, ....For k 2 1, the surface boundary condition is (d/dz)g,(A, O) = O and all solutions arerequired to decay to zero as z 4 CO. For k = 1, $,(A, z) = $(A, z) is the field due tothe unperturbed system. We note that each of the problems in (33) has the sameform and is identical to that given in (24). $,(A, O) can be found using the adjointGreen's function approach and hence the Fréchet derivative f(')(A, 0)6w = &$,(A, O)will again be given by (30). Higher-order Fréchet derivatives may be calculatedusing the same adjoint Green's function.(33)Riccati equation approachThe third method for computing the Fréchet derivative is less general than theadjoint method in that it is restricted to governing equations of the formUyz) + A(z)u'(z) + B(z)u(z) = O, (34)that is, to second-order homogeneous linear equations in one-dimension. Nevertheless,the pervasiveness of this equation in physical problems means that special solutionmethods tied to this equation can still have general usefulness (e.g. Oldenburg1978 and 1979). To compute the Fréchet derivative, we first transform (34) toa Riccati equation by making the substitution y(z) = - ur(z)/a(z)u(z) ory(z) = -a(z)u(z)/u'(z) (Bender and Orszag 1978). The Riccati equation for ay,obtained by perturbing the model, is then solved using standard techniques.To illustrate the method, we apply it to the 1D resistivity problem. FollowingOldenburg (1978) let S(A, z) = h(A, z)/h'(Â, z). Substituting into (22a) yields theRiccati equationdS- + w(z)S + Â2S2 - 1 = o.dz(35)Perturbing w(z) by 6w(z) generates a response perturbation 6S(A, z) which satisfies,to first order, the equationd6S- + (w(z) + 2A2S)dS = -SGw(z).dz(36)

CALCULATING FRÉCHET DERIVATIVES 507This may also be written as(37)Making use of the integrating factor(38)we obtain an expression for the Fréchet derivative S(')(A, O) given byThis is identical to that obtained using the adjoint Green's function approach forthe $(A, O) response.The Riccati-equation approach is particularly interesting because it allows oneto compute Fréchet derivatives for a response or for its reciprocal. For example, wecould have chosen the responseThis leads to the Riccati equationdR--dzw(z)R + R2 - 1' = O, (41)and to a Fréchet derivative R(')(A, O) given byAt the surface of the earth, either S(A, O), or its reciprocal R(A, O), is measured.For use in a linearized inverse solution one would like to choose the datum and itscorresponding Fréchet derivative which leads to the most linear problem possible.This choice of data is also encountered in electromagnetic problems where bothelectric (E) and magnetic (B) fields are measured and their ratio, either E/B or B/Ecan be regarded as data. An analysis, making use of the Fréchet derivative for bothdata choices, can resolve which datum should be used for linearization.PARAMETERIZATION OF THE FORWARD AND INVERSE PROBLEMSIn a parametric formulation of a forward or inverse problem, the model can bewritten as

508 P. R. McGILLIVRAY AND D. W. OLDENBURGwhere {$k} is a set of basis functions. The choice of basis functions determines thetype of model to be considered. For example, the model domain D can be dividedinto subdomains Dk with the kth basis function defined to be unity over the kthsubdomain and zero elsewhere. The kth parameter mk is then the value of the modelover the corresponding subdomain. This parameterization permits ‘blocky’ modelswhich are well suited to problems where large changes in the model are expectedover short distances. Another approach is to define a grid of nodes over D and to letthe kth parameter be the value of m(x) at the location of the kth node. {!),’I isdetermined by the interpolation method used to define the model between nodes. Athird possibility is to choose {$k} as a set of regional basis functions (e.g. sinusoidalfunctions used in a Fourier expansion).Regardless of the parameterization selected, the model is completely specified interms of the vector m = (mi, m, ,..., mM). In solving the inverse problem, the goal isto find values for these parameters which will acceptably reproduce the observations.Although there are many ways in which this problem can be approached - forexample, using the steepest descent, conjugate gradient, or Gauss-Newton method(Gill, Murray and Wright 1981) - the inversion will generally require the computationof the sensitivity or partial derivative matrix in (10).When the forward problem is expressed as a boundary-value problem, the sensitivitiescan be computed by solving another boundary-value problem which is eitheridentical, or closely related to, the original problem. Many standard algorithms areavailable for the numerical solution of boundary-value problems - the most commonlyused being the finite-difference, finite-element and boundary-element methods(e.g. Lapidus and Pinder 1982). These algorithms make use of a discretization toreduce the governing differential equation and boundary conditions to a system ofalgebraic equations. For a general forward problem, this system can be expressed asAU = q, (44)where U is the solution at a set of discrete points in the domain. The entries of thecoefficient matrix A depend on the material properties of the model and on thediscretization scheme adopted. This means that a new coefficient matrix must beformed each time the forward problem is to be solved using a new model. Theentries of the vector q, on the other hand, are independent of the material propertiesof the model and depend only on the source distribution and discretization scheme.Because of the need to solve (44) for many different source distributions, a directsolver based on some kind of factorization of A is generally used to obtain U. Onepossible factorization is given byA=LU (45)where L and U are respectively lower and upper triangular matrices (Golub andVan Loan 1983). The system in (44) becomesLUU = q. (46)Letting Uu = v, the vector v can be computed by solving Lv = q by forward substitution;U may be recovered by solving Uu = v by back substitution. Once the factors

CALCULATING FRÉCHET DERIVATIVES 509of A have been computed, they can be used repeatedly to solve the forward problemfor different source configurations at little additional expense.CALCULATION OF DIFFERENTIAL SENSITIVITIESGiven that the forward problem can be expressed as a boundary-value problem,there are three ways to obtain the sensitivities. In the first method, the sensitivitiesare computed from their finite-difference approximations, each requiring the solutionof the forward problem with the corresponding parameter slightly perturbed. Inthe second method, a new boundary-value problem is derived for each of the sensitivities,and the sensitivities are solved for directly. In the third, the sensitivities arecomputed using the solution to an adjoint Green’s function problem.Perturbation approachThe most straightforward way to calculate the differential sensitivities is toapproximate them using the one-sided finite-difference formulaaFJ(m) FXm + Amk) - F,(m)xAmkThe perturbed forward response FJ(m + Amk) is obtained by re-solving the forwardproblem after the kth parameter has been perturbed by an amount Amk. Since themodel must be altered to compute the perturbed responses, each sensitivity requiresthe solution of a completely new problem. As such, this ‘brute force’ method isinefficient, but it can nevertheless yield useful results (e.g. Edwards, Nobes andGomez-Trevifio 1984).Sensitivity equation approachIn the sensitivity-equation method, a new forward problem is derived whosesolution is the desired sensitivity function 4k(x). Problems which have beenaddressed using this approach include the 2D magnetotelluric problem (Rodi 1976;Jupp and Vozoff 1977; Cerv and Pek 1981; Hohmann and Raiche 1988), the 2Delectromagnetic problem (Oristaglio and Worthington 1980) and computer-aideddesign problems (Brayton and Spence 1980). Vemuri et ai. (1969), McElwee (1982)and Townley and Wilson (1985) use the approach to address problems in groundwaterflow.To illustrate the technique, we consider the steady-state diffusion problem givenin (3). Taking p(x) to be the model, and assuming the parameterization p(x) =pl Jll(x), we obtainIf”=(47)aucr(x)u + p(x) - = Oanon aD.(48b)

510 P. R. McGILLIVRAY AND D. W. OLDENBURGDifferentiating (48) with respect to pk , and substituting for uk(x) = au(x)/ap,, yieldsthe sensitivity problem-V . (p(x)vuk) + q(x)uk = V * (t,bk(x)Vu) in D, (494auka(x)uk + B(x) - = Oanon aD.To compute the sensitivities for a model p(x), the forward problem (3) is firstsolved to obtain u(x) at all points x in D. For each parameter pk, the correspondingsensitivity problem is solved to obtain uk(x) which is then evaluated at each of theobservation locations. Note that the source term V . (t,bk(x)Vu) differs for each k, sothat a total of M + 1 forward problems must be solved to obtain all of the sensitivities.Since the sensitivity problems and the original forward problem differ only interms of their right-hand sides, they can be solved efficiently if a direct method suchas the LU decomposition is used. Only a single matrix factorization is then requiredto solve all M + 1 forward problems.The sensitivity-equation method is easily extended to the calculation of otherkinds of sensitivities. For example, the directional sensitivity u,(x), given bywhere a is a unit vector in parameter space, can be computed by first multiplying(49) by ak and then summing over k to obtain the new problem-v . (p(x)vu,) + q(x)u, = v . ( 5 ak $&)vu) in D, (514k=laua(x)u, + B(x) 2 = 0anon ôD.u~,(x) may then be solved for directly. Directional sensitivities are useful for determiningan optimum model perturbation once a direction for the perturbation hasbeen selected (e.g. Townley and Wilson 1985).The sensitivity-equation method can also be extended to the calculation ofhigher-order sensitivities. For example, differentiating (49) with respect to the newparameter pi yields a new problem whose solution is the second-order sensitivityfunction ukl(x) = a2u(x)/apk ûpi . Several parameter-estimation schemes that makeuse of these higher-order sensitivities to achieve rapid convergence are available (e.g.Brayton and Spence 1980; Gill et al. 1981). The use of second-order sensitivities inthe estimation of parameter uncertainty has also been described (Townley andWilson 1985).Adjoint equation approachThe third method for calculating sensitivities is based on the adjoint Green’sfunction concept discussed earlier. Some of the problems to which the adjointequationapproach has been applied include the seismic problem (Tarantola 1984;

CALCULATING FRÉCHET DERIVATIVES 51 1Chen 1985), the resistivity problem (Smith and Vozoff 1984), the magnetotelluricproblem (Weidelt 1975; Park 1987), the groundwater flow problem (Neuman 1980;Carrera and Neuman 1984; Sykes and Wilson 1984; Sykes Wilson and Andrews1985; Townley and Wilson 1985), the reservoir evaluation problem (Carter et al.1974), and various problems in computer-aided design (Director and Rohrer 1969;Branin 1973; Brayton and Spence 1980). An equivalent approach, based on a reciprocityrelationship for transmission networks, has also been described (Madden1972; Tripp, Hohmann and Swift 1984).As an illustration of the approach, consider again the steady-state diffusionproblem given in (3). Having obtained the sensitivity problem in (49), the appropriateadjoint problem is found by constructing an operator L* and boundary conditionssuch that (17) is satisfied. We obtainL*G* = -V . (p(x)VG*) + q(x)G* = 6(x - xj) in D, (524dG*a(x)G* + p(x) - = OdnForming the expressionon dD.(52b)G*LUk - uk L*G* = G*V ’ (l,!/k VU) - uk 6(X - Xj), (53)and integrating over D yieldsG*V ’ (l,!/k VU) d32. (54)To compute the sensitivities, (52) must be solved for each observation locationxi. The integration in (54) is then carried out for each parameter. Since the sourceterm in (52a) differs for each observation location, a total of N + 1 forward problemsmust be solved (although again this can be done efficiently if an LU decompositionor other factorization is used).In some cases, the sensitivity of an objective function is desired. For example, if asteepest-descent method is to be used to minimize the objective function4(m) =Nj= 1[eTbs - ej]’where ej = F,{m), then sensitivities of the formN-84= -2 1 (e:” - ej)uk(xj)j= 1must be calculated. Although this could be done by solving for each U&),practical method can be arrived at by solving the modified adjoint problemL*G* = - V . (p(x)VG*) + q(x)G* = - 2 1 (egbs - ej)6(x - xj)aG*a(x)G* + p(x) - = Odnon do.Nj= 1in D,(55)a more(57a)(57b)

512 P. R. McGILLIVRAY AND D. W. OLDENBURGThe objective function sensitivities can then be computed fromUsing this approach only two forward problems need be solved to obtain allrequired sensitivities.EXAMPLE : 1 D RESISTIVITY PROBLEMTo illustrate the numerical computation of sensitivities, we consider the 1D resistivityproblem given in (22). Let m(z) = In p(z) be the model and represent the earthby a sequence of layers of constant conductivity. Thenwhere1 for z1 < z < z ,+~O otherwiseand zl is the depth to the top of the lth layer.The reference model for this example, shown in Fig. la, consists of a 100 Rmconductive zone buried within a more resistive 1000 Rm half-space. The intervalbetween z = O m and z = 400 m was divided into 20 m thick layers and the logresistivities of all but the first layer were taken to be parameters. (The log resistivityof the surface layer was not considered to be a parameter since if it were, the dependenceof the boundary condition (22b) on the surface conductivity would unnecessarilycomplicate the example.) The transformed potentials for different values of 1160 -hE 140->vv 2n 120-r 100-80 - bFIG. 1. (a) Conductivity model used in the 1D resistivity example. (b) Transformed surfacepotential h(l, O) computed for the model shown in (a).

CALCULATING FRÉCHET DERIVATIVES 513were computed using the propagator matrix algorithm described in Appendix A.Those results are shown in Fig. lb. The presence of the conductive zone is indicatedby the anomalously low value of the transformed potential.Taking the transformed potential for 1 = 0.005 m-' to be the single datum forthis example, the differential sensitivities hk(II, O) = ùh(II, û)/ûmk were computed usingeach of the three methods described in this paper.First, making use of the perturbation approach and the approximation given in(47), the sensitivities for each layer were computed using conductivity perturbationsof 1.0%, 10.0% and 100.0%. The sensitivities were computed using different perturbationsso that the effect of perturbation size on the accuracy of the approximationcould be examined. The results, plotted as functions of depth, are shown in Fig. 2a.As expected, the transformed potential is found to be most sensitive to changes inthe near-surface conductivity, and a rapid decrease in sensitivity is observed at thetop of the conductive zone. The shielding effect of the conductive layer is also apparent,resulting in the transformed potential being insensitive to changes in the conductivitybelow the layer. A comparison of the sensitivities obtained for the threedifferent perturbation steps indicates that this approach is reasonably accurateexcept when large perturbations are used. The computational work involved,however, is great for problems that involve a large number of parameters relative tothe number of data available. For this example a total of 20 forward problems hadto be solved to obtain the sensitivities for only one datum.The sensitivity-equation formulation is obtained by differentiatingrespect to mk. This yieldsd2hk dhk dlC/k dh dh-- W(Z) -- A2hk =--- = [6(Z - Zk-l) - 6(Z - Zk)] -,dz2 dz dz dz dzd- hk(A, O) = O,dzThe current source represented by the non-homogeneous boundary condition (22b)in the original problem must be replaced by two buried sources - one on either sideof the kth layer. Although dhldz is discontinuous at each of the layer boundaries,the right hand side of (60a) can still be evaluated usingdh 1 dh6(z - q) - = - 6(z - ZJdz 2The sensitivity function hk(l, z) is obtained by using the propagator matrix algorithmto solve (60) for each layer. The sensitivity-vs-depth relationship is shown inFig. 2b. A total of 20 forward problems again had to be solved to compute thesensitivities for the one value of II. Comparing Figs 2a and 2b, it is found that thesensitivities computed using this method correspond to those found by the perturbationapproach for Aa/a -+ O.

514 P. R. McGILLIVRAY AND D. W. OLDENBURGh20EU > 15OO 100 200 300 400Depth (rn)FIG. 2. Sensitivity as a function of depth for Â. = 0.005 m-' computed using (a) the perturbationmethod, (b) the sensitivity-equation method, and (c) the adjoint-equation method.To solve for the sensitivities using the adjoint-equation approach, we require thesolution of the adjoint problem- dZG*+ ddz2- dz(wG*)- AZG* = 6(~),d- G*(Â, O) + w(O)G*(A, O) = O,dz

CALCULATING FRÉCHET DERIVATIVES 515G*(Â, z) = O. (624The sensitivity hk(Â, O) is then given byThe adjoint problem can also be solved (after some manipulation) using thepropagator matrix algorithm, although a simple relationship can be found whichallows G*(Â, z) to be calculated directly from h(Â, z). Note that if a new functionG* = G*p is defined, then G*(Â, z) is found to satisfy the same forward problem ash(Â, z) except that the current source is scaled by - ÂI/27c.’ Since the original governingdifferential equation is linear, the Green’s function G*(Â, z) required in (63) canbe related to h(Â, z) by27c h(Â, z)G*(Â, Z) = - - -.11 P(4Once the forward problem has been solved for h(Â, z), then (64) is used to obtainG*(Â, z) and the integrations in (63) are carried out to compute the sensitivities. Theresults, shown in Fig. 2c, are identical to those found using the sensitivity-equationmethod. Only a single forward solution is required to generate all of the sensitivitiesusing this approach.EXAMPLE : 2D RESISTIVITY PROBLEMTo illustrate the sensitivity and adjoint equation methods for a more complicatedsituation, consider the 2D resistivity problem described by the differential equationand the boundary conditions4(% Y, Z)Lm = O, (654where 4(x, y, z) is the potential due to a single current electrode located at x, = (x, ,O, z,) and R = [(x - x,)’ + y’ + (z - z,)’]~/’. The second current electrode isassumed to be located at infinity. Because of the 2D nature of the conductivitystructure, it is convenient to consider the Fourier cosine transform of 4 given by

516 P. R. McGILLIVRAY AND D. W. OLDENBURGThe transformed response 6 then satisfiesa- 6 (x7 Ky 7 z, laD = O7an(67b)6(X7 Ky, Z)~F-~ = 07(674where r = [(x - x,)’ + (z - z,)’]’/’.The forward problem given in (67) can be solved using an integrated finitedifferencealgorithm (Narasimhan and Witherspoon 1976; Dey and Morrison 1979)to yield transformed potentials over a finite-difference mesh for any particular valueof Ky. The forward responses in the spatial domain can be obtained by numericallyevaluating the inverse cosine transformNumerical computations proceed by representing the conductivity model a(x, z) bya set of rectangular prisms of constant conductivity which extend to infinity in they-direction. The parametrized model is then given bywhereMz Mxa(x? z, = 1 1 ajk$jk(x7 z)7 (69)j=i k=lThe sensitivity problem, obtained by differentiating (67) with respect toisa- an 6jdX, Ky 9 z, laD = O,(70b)6jk(X,Ky, z, Ir-m = O,(704where $jk(X, Ky z) = a /aajk 6(x7 Ky z) is the desired sensitivity in the transformeddomain.The adjoint Green’s function problem corresponding to (70) is

CALCULATING FRÉCHET DERIVATIVES 517Since the form of (71) is identical to that of the original transformed forwardproblem in (67), the same finite-difference algorithm can be used to solve both. Thistakes advantage of the computational savings associated with decomposing theoriginal matrix.Note that for the 2D resistivity problem it will often be the case that an electrodeused to make one or more potential measurements will have also been used atsome point in the survey as a current electrode. If this occurs, then (71) need not besolved since G*(x, Ky z7 xo , zo) can always be computed directly from the forwardmodelled response $(x, Ky, z) due to the current electrode at (xo , zo).Once the adjoint Green’s function has been obtained, the sensitivity of the correspondingtransformed potential can be computed fromG*(x, Ky, z, xo , zo) d2x. (72)An inverse cosine transform is then required to obtain the sensitivity of the potentialin the spatial domainAs an illustration of the adjoint-equation method we compute the sensitivities for ahalf-space of 1000 Rm. For this example we concentrate on a limited region of themodel (i.e. -200 m < x < 200 m and 20 m < z < 200 m). The parametrization ofthis region is indicated on the cross-sections shown in Figs 3 and 5. Having placed acurrent electrode at x, = (125, O, O) m and a potential electrode at xo = (- 125, O,O) m, the sensitivity of the measured potential to changes in the conductivity of eachprism was computed. A total of two forward solutions for each of the nine Ky valuesused in the inverse transform were needed to obtain all 312 sensitivities. The absolutevalue of the resulting sensitivities, displayed in cross-section form, are shown inFig. 3a. These numerically-generated sensitivities compare well with those computedfrom the analytic solutions for the potential and Green’s function, shown in Fig. 3b.We see from the results for the half-space that the sensitivity is largest at thoseblocks which are near to either of the electrodes. We notice also that the sensitivitycan be either positive or negative. There is a region af positive sensitivity at shallowdepths in a semicircle between the current and potential electrode; outside thisregion the sensitivity is negative which means that the potential at the measuringsite will decrease when the conductivity increases.

518 P. R. McGILLIVRAY AND D. W. OLDENBURG- cO r O m m9 ? 9?N2 O 2LDX

CALCULATING FRÉCHET DERIVATIVES 519I65m1FIG. 4. Cross-section showing a 100 Rm conductive prism buried in a loo0 Rm halfspace.In any inverse problem, one attempts to acquire data which provide complementaryinformation about the earth structure. In the d.c. resistivity problem, we expectdifferent sensitivities if the potential is measured at a different offset in either the x-or y-direction. The sensitivities resulting from changing the offset in the y-directionor strike direction aie particularly easy to compute. Having obtained the transformedsensitivity 6, for enough values of Ky to allow for an accurate numericalevaluation of (73), one can compute the sensitivity of potentials for different pointsalong the strike direction by simply re-evaluating the inverse transform using differentvalues of y,. Thus the sensitivities for off-line potentials can be obtained at littleadditional expense once the on-line sensitivities have been computed. Fig. 3c showsthe sensitivity computed for a potential electrode located at x,, = (- 125,200, O) m.As a final example, the adjoint method was used to compute the sensitivities forthe more complicated situation of a conductive body buried in a half-space (Fig. 4).The sensitivities for the same current and on-line potential electrode as in Fig. 3 areshown in Fig. 5. There has been considerable change in the sensitivity patterncaused by the conductive prism. Overall, the amplitudes of the sensitivities haveincreased and the demarcation between positive and negative sensitivities has beengreatly distorted in the region between the two electrodes where the prism lies.FIG. 3. Absolute sensitivities computed for a loo0 Rm half-space using (a) the adjointequationapproach, and (b) the analytic expressions for the potential and Green’s function.The current and potential electrodes were located at x, = (125, O, O) m and x,, = (- 125, O, O) mrespectively. (c) Absolute sensitivities computed using the adjoint-equation approach for anoff-line potential electrode at xo = (-125, 200, O) m. The dashed line is the demarcationbetween positive and negative values of the sensitivities.

~520 P. R. McGILLIVRAY AND D. W. OLDENBURG201.260.80hEv5anQ1 O0h>v0.63>i.- c.- >.- cm0.44 C0.3 1200N IdAOX (rn)FIG. 5. Absolute sensitivities computed for the conductive prism model shown in Fig. 4 usingthe adjoint-equation approach.dOO N0.22SUMMARYThe efficient solution of the non-linear inverse problem requires that the dependenceof the data on changes in the model be easily quantified. The most convenient wayof quantifying this dependence is to derive the analytic Fréchet derivative using oneof the three procedures described in this paper. In most cases, however, it is notpossible to derive an analytic expression for either the forward responses or theFréchet derivatives, so one must resort to a model parameterization to simplify theproblem. The dependence of the data on changes in the model is then given by a setof partial derivatives or sensitivities. These sensitivities can be computed from theirfinite-difference approximation, although this requires the solution of a largenumber of forward problems. When dealing with boundary-value problems we canderive a new set of equations whose solution yields either the sensitivities themselvesor quantities from which the sensitivities can be calculated. The similarity of thesenew equations and the original forward problem allows the sensitivities to be computedefficiently if a numerical solution based on a matrix factorization is used.ACKNOWLEDGEMENTSThe authors wish to thank S. Dosso and U. Haussman for helpful discussions concerningthe series-expansion approach to the calculation of Fréchet derivatives. Weare also grateful to the two anonymous referees who provided many constructivesuggestions for this paper.

CALCULATING FRÉCHET DERIVATIVES 521APPENDIX AForward responses for the ID resistivity problemLet the earth be represented by a sequence of M, layers of constant conductivityover a half-space. Within any of the layers or the half-space, the governing differentialequation for the 1D resistivity problem reduces to-_ d2hdz2Â2k = O.The general solution to (Al) for the kth layer can then be written ash(Â, .) = Uke-"Zk-l-Z) + Dked(Zk-i-z) >('42)and for the half-space, ask(1, z) = D, + 1Making use of the continuity relationshipk(Â, z:) - k(Â, zk) = O,('43)and the conservation of charge relationship-- bk+lOkk(Â, z:) --- k(Â, z;) =1 dz 1 dz2n for k = k,O otherwise,one can solve for the coefficients U and D for the kth layer in terms of those for the(k + 1)th layer. This leads to the propagator matrix expressionwhereandThe source vector s k is given byfor k = k.

522 P. R. McGILLIVRAY AND D. W. OLDENBURGCombining the propagator matrix expressions for each layer yieldsU, and D, can be related using the boundary condition at z = O which requires thath(l, z) satisfyc- Z for surface electrode,--- 01 dh(A, O+) = -0l(U1 - DI) =Â. dzO for buried electrode.The solution h(Â., O) = U, + D, is then obtained from (A6) and (A7).REFERENCESBENDER, C.M. and Orszag, S.A. 1978. Advanced Mathematical Methods for Scientists andEngineers. McGraw-Hill Book Co.BRANIN, F.H., Jr. 1973. Network sensitivity and noise analysis simplified. ZEEE Transactionson Circuit Theory CT-20,285-288.BRAYTON, R.K. and SPENCE, R. 1980. Sensitivity and Optimization. Elsevier Science PublishingCo.CARRERA, J. and NEUMAN, S.P. 1984. Adjoint state finite element estimation of aquifer parametersunder steady-state and transient conditions. In: Proceedings of the 5th InternationalConference on Finite Elements in Water Resources. Springer-Verlag, Inc.CARTER, R.D., KEMP, L.F., JR, PIERCE, A.C. and WILLIAMS, D.L. 1974. Performance matchingwith constraints. Society of Petroleum Engineering Journal 14, 187-196.CERV, V. and PEK, J. 1981. Numerical solution of the two-dimensional inverse geomagneticinduction problem. Studia Geophysica et Geodaetica 25,69-80.CHAVE, A.D. 1984. The Féchet derivatives of electromagnetic induction. Journal of GeophysicalResearch 89, 3373-3380.CHEN, Y.M. 1985. Generalized pulse-spectrum technique. Geophysics JO, 1664-1675.DEY, A. and MORRISON, H.F. 1979. Resistivity modelling for arbitrarily shaped twodimensionalstructures. Geophysical Prospecting 27, 106136.DIRECTOR, S.W. and ROHRER, R.A. 1969. The generalized adjoint network and network sensitivities.IEEE Transactions on Circuit Theory CT-16, 313-323.EDWARDS, R.N., NOBES, D.C. and G~MEz-TREVINO, E. 1984. Offshore electrical exploration ofsedimentary basins : The effects of anisotropy in horizontally isotropic, layered media.Geophysics 49, 566576.GILL, P.E., MURRAY, W. and WRIGHT, M.H. 1981. Practical Optimization. Academic Press,Inc.GOLUB, G.H. and VAN LOAN, C.F. 1983. Matrix Computations. Johns Hopkins UniversityPress.GRIFFEL, D.H. 1981. Applied Functional Analysis. Ellis Horwood Limited.HOHMANN, G.W. and RAICHE, A.P. 1988. Inversion of controlled source electromagnetic data.In: Electromagnetic Methods in Applied Geophysics, Vol. 1, Theory, M. Nabighian (ed.).Society of Exploration Geophysics.

CALCULATING FRÉCHET DERIVATIVES 523JACKSON, D.D. 1972. Interpretation of inaccurate, insufficient and inconsistent data. GeophysicalJournal of the Royal Astronomical Society 28,97-110.JUPP, D.L.B. and VOZOFF, K. 1975. Stable iterative methods for the inversion of geophysicaldata. Geophysical Journal of the Royal Astronomical Society 42,957-976.JUPP, D.L.B. and VOZOFF, K. 1977. Two-dimensional magnetotelluric inversion. GeophysicalJournal of the Royal Astronomical Society 50, 333-352.LANCZOS, C. 1960. Linear Diyerential Operators. D. Van Nostrand.LAPIDUS, L. and PINDER, G.F. 1982. Numerical Solutions of Partial Diflerential Equations inScience and Engineering. John Wiley and Sons, Inc.MADDEN, T.R. 1972. Transmission systems and network analogies to geophysical forwardand inverse problems. Report 72-3, Department of Earth and Planetary Sciences, MIT.MCELWEE, C.D. 1982. Sensitivity analysis and the ground-water inverse problem. Groundwater20, 723-735.MENKE, W. 1984. Geophysical Data Analysis: Discrete inverse Theory. Academic Press, Inc.NARASIMHAN, T.N. and WITHERSPOON, P.A. 1976. An integrated finite difference method foranalyzing fluid flow in porous media. Water Resources Research 12, 57-64.NEUMAN, S.P. 1980. Adjoint-state finite element equations for parameter estimation. In: Proceedingsof the Third international Congress on Finite Elements in Water Resources, 2.662.75. University of Mississippi.OLDENBURG, D.W. 1978. The interpretation of direct current resistivity measurements. Geophysics43,61&625.OLDENBURG, D.W. 1979. One-dimensional inversion of natural source magnetotelluric observations.Geophysics 44, 1218-1244.OLDENBURG, D.W. 1984. An introduction to linear inverse theory. IEEE Transactions onGeoscience and Remote Sensing GE-22,665-674.ORISTAGLIO, M.L. and WORTHINGTON, M.H. 1980. Inversion of surface and borehole electromagneticdata for two-dimensional electrical conductivity models. Geophysical Prospecting28, 633-657.PARK, S.K. 1987. Inversion of magnetotelluric data for multi-dimensional structures. Institutefor Geophysics and Planetary Physics. Report 87/6, University of California.PARKER, R.L. 1977a. The Fréchet derivative for the one-dimensional electromagnetic inductionproblem. Geophysical Journal of the Royal Astronomical Society 49, 543-547.PARKER, R.L. 1977b. Understanding inverse theory. Annual Reviews of Earth and PlanetarySciences 5, 35-64.ROACH, G.F. 1982. Green’s Functions. Cambridge University Press.RODI, W.L. 1976. A technique for improving the accuracy of finite element solutions for MTdata. Geophysical Journal of the Royal Astronomical Society 44,483-506.SMITH, N.C. and VOZOFF, K. 1984. Two-dimensional DC resistivity inversion for dipoledipoledata. IEEE Transactions on Geoscience and Remote Sensing GE-22,21-28.SYKES, J.F. and WILSON, J.L. 1984. Adjoint sensitivity theory for the finite element method.In: Proceedings of the Fifth international Congress on Finite Elements in WaterResources, 3-12.SYKES, J.F., WILSON, J.L. and ANDREWS, R.W. 1985. Sensitivity analysis for steady stategroundwater flow using adjoint operators. Water Resources Research 21, 359-371.TARANTOLA, A. 1984. Linearized inversion of seismic reflection data. Geophysical Prospecting32,998-1015.TOWNLEY, L.R. and WILSON, J.L. 1985. Computationally efficient algorithms for parameterestimation and uncertainty propagation in numerical models of groundwater flow.Water Resources Research 21.1851-1860.

524 P. R. McGILLIVRAY AND D. W. OLDENBURGTRIPP, A.C. HOHMANN, G.W. and SWIFT, C.M., JR. 1984. Two-dimensional resistivity inversion.Geophysics 49, 1708-1717.VEMURI, V., DRACUP, J.A., ERDMANN, R.C. and VEMURI, N. 1969. Sensitivity analysis methodof system indentification and its potential in hydrologic research. Water 'ResourcesResearch 5,341-349.WEIDELT, P. 1975. Inversion of two-dimensional conductivity structures. Physics of the Earthand Planetary Interiors 10,282-291.WIGGINS, R.A. 1972. The general linear inverse problem: Implication of surface waves andfree oscillations for Earth structure. Reviews of Geophysics and Space Physics 10, 251-285.ZEIDLER, E. 1985. Nonlinear Functional Analysis and its Applications III - VariationalMethods and Optimization. Springer-Verlag, Inc.