The nonlinear residual calculation uses the same vector decomposition used in thematrix{vector and Jacobian routines. Hence, no data needs to be transferred betweenprocessors to start this procedure.One aspect <strong>of</strong> the Jacobians having so many nonzeroes is that we can aord to storethe complete solution vector on a single processor without using much extra memory.In our case, our parallel computers had from 128 Mb to 2 Gb <strong>of</strong> local memory on eachnode. Hence, if some operation is quick and does not parallelize well (or at all), we canactually gather all <strong>of</strong> the data, do the operation on a single node, and then scatter databack to the remaining processors. This was particularly useful while parallelizing thecode one major operation at a time and for debugging purposes.The general computing methodology used was that there was one processor thatdirected all <strong>of</strong> the processors to do various computing tasks. These included havingprocessors do their part <strong>of</strong> the parallel iterative procedures, compute a Jacobian, orevaluate a nonlinear residual. Large tasks are the typical operation, not small tasks likerequiring all processors to do an inner product.The communications library we used, EUIH [6], allowed us to produce one executablefor all <strong>of</strong> the following IBM computers: SP1, SP2, and clusters <strong>of</strong> IBM RS-6000's. No relinking or recompiling was necessary in order to use the fast switches.Hence, only one copy <strong>of</strong> the source les and one executable were maintained for all <strong>of</strong>these environments.NUMERICAL EXPERIMENTSThe ame sheet model provides a test <strong>problem</strong> <strong>of</strong> a moderate computational costwith which to probe the eciency <strong>of</strong> solution algorithms that can then be used totackle detailed chemistry diusion <strong>problem</strong>s. Such a study is presented in [2] [3]. Theame sheet that is considered is adequately resolved on a grid with up to 210 4 nodes,with 20% <strong>of</strong> the mesh nodes clustered in a region covering 0.1% <strong>of</strong> the computationaldomain. This requires up to 100 megabytes <strong>of</strong> work space. The adaptive mesh multigridprocedure outlined above is particularly ecient for solving ame sheet <strong>problem</strong>s. Incomparison to traditional solution procedures, the total execution times drop by a factor<strong>of</strong> 10. Speedups as high as 166 in the time relaxation phase have been seen on a singleprocessor. For three dimensional <strong>problem</strong>s, speedups much greater than 10 should beobtained. At the beginning <strong>of</strong> this study, wespentasmuch as96minutes on the amesheet phase using an IBM RISC System/6000 model 560 workstation. Recently, wespent as little as 45 seconds using an eight node SP2.Solving a nite rate chemistry ame <strong>problem</strong> instead <strong>of</strong> a ame sheet <strong>problem</strong>is a quite challenging step which involves a dramatic increase in the diculty <strong>of</strong> the<strong>problem</strong>. This is mostly attributable to the much larger number <strong>of</strong> dependent unknowns,the nonlinear uid dynamics-thermochemistry coupling, and the disparate length scalesthat must be resolved in the computed solution. In particular, excellent resolution <strong>of</strong>a ame sheet <strong>problem</strong> requires up to 128 megabytes, while a reasonable solution <strong>of</strong>a diusion ame with 50 chemical species may require up to a gigabyte. Traditionalsolvers for two-dimensional <strong>laminar</strong> diusion ames used to require more than a hundred8
CPU hours on a supercomputer to get a solution. Thanks to recent developments <strong>of</strong>computational algorithms and computer capabilities, this is no longer the case.The ame conguration is a methane-air lifted <strong>laminar</strong> diusion ame with a tripleame structure at its base, as illustrated in Figure 1. Although extremely dicult tocompute <strong>numerical</strong>ly, this ame conguration is chosen in the present study since experimentaland <strong>numerical</strong> results are already available for these ames [8]. The <strong>numerical</strong>solution that is considered rst includes 16 chemical species engaged in a C1-chain reactionmechanism, i.e., only molecules with at most one carbon atom are considered. Inaddition to the 16 chemical species, 4 unknowns are associated with each mesh point.Hence, there are a total <strong>of</strong> 20 unknowns per mesh point. The ame is appropriatelyresolved using 510 3 mesh nodes, and very good agreement with previous experimental(2:510 5 data points) and <strong>numerical</strong> data is obtained.Consider a one way nonlinear multigrid with Bi{CGSTAB/GS as the solver on alllevels. On the nest level, these are the (wall clock) times we see to converge on the89 85 ne grid.Machine Processors MinutesRISC System/6000{580 1 602.59SP1 4 137.218 70.3712 58.8916 53.85SP2 4 47.416 30.05Peak oating point is about 130 Mops for the SP1 and RISC System/6000{580, butabout 266 for the SP2.The chemical mechanism just considered yields adequate resolution for the temperatureand several chemical species. More detailed investigation <strong>of</strong> the ame structurerequires additional chemical species in the model. For instance, the study <strong>of</strong> pollutantformation, such as nitric oxide, requires up to 50 chemical species. While thememory requirements are too large for current serial computers, such <strong>problem</strong>s can beadequately solved on parallel computers with a large amount <strong>of</strong> memory per node. Thediusion ame just described can then be conveniently used to initialize diusion amecalculations with more detailed chemistry submodels. Such work is in progress.REFERENCES[1] C. C. Douglas, Implementing abstract multigrid or multilevel methods, in Sixth Copper MountainConference on Multigrid Methods, N. D. Melson, T. A. Manteuel, and S. F. McCormick, eds.,vol. CP 3224, Hampton, VA, 1993, NASA, pp. 127{141.[2] C. C. Douglas and A. Ern, Numerical solution <strong>of</strong> ame sheet <strong>problem</strong>s with and withoutmultigrid methods, in Sixth Copper Mountain Conference on Multigrid Methods, N. D. Melson,T. A. Manteuel, and S. F. McCormick, eds., vol. CP 3224, Hampton, VA, 1993, NASA,pp. 143{157.9