Neural Network Based Wind Vector Retrieval from Satellite ...
Neural Network Based Wind Vector Retrieval from Satellite ...
Neural Network Based Wind Vector Retrieval from Satellite ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
DeptofComputerScience&AppliedMathematics <strong>Neural</strong>ComputingResearchGroup BirminghamB47ET AstonUniversity<br />
http://www.ncrg.aston.ac.uk/ Fax:+44(0)1213334586 Tel:+44(0)1213334631 UnitedKingdom<br />
<strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong> <strong>Neural</strong><strong>Network</strong><strong>Based</strong><br />
<strong>Satellite</strong>ScatterometerData<br />
TechnicalReportNCRG/99/003 DanCornford,IanT.Nabney,ChristopherM.Bishopa d.cornford@aston.ac.uk<br />
aMicrosoftResearch,1GuildhallStreet,CambridgeCB23NH,UK Accepted<strong>Neural</strong>ComputingandApplications January26,1999<br />
Obtainingwindvectorsovertheoceanisimportantforweatherforecastingandoceanmodelling. Severalsatellitesystemsusedoperationallybymeteorologicalagenciesutilisescatterometersto inferwindvectorsovertheoceans.Inthispaperwepresenttheresultsofusingnovelneural Abstract<br />
networkbasedtechniquestoestimatewindvectors<strong>from</strong>suchdata.Theproblemispartitionedinto (MLP)andasumofsquareserrorfunction.<strong>Wind</strong>directionisaperiodicvariableandamulti- estimatingwindspeedandwinddirection.<strong>Wind</strong>speedismodelledusingamulti-layerperceptron thefullperiodicprobabilitydensityofdirectionconditionedonthesatellitederivedinputsusing valuedfunctionforagivensetofinputs;aconventionalMLPfailsatthistask,andsowemodel aMixtureDensity<strong>Network</strong>(MDN)withperiodickernelfunctions.Acommitteeoftheresulting MDNsisshowntoimprovetheresults.<br />
Keywords:ConditionalProbabilityDensityEstimation,MixtureDensity<strong>Network</strong>,MultiLayer Perceptron,PeriodicVariables,<strong>Wind</strong>vectors,Scatterometer.
2 1 Introduction <strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData<br />
ObtainingwindvectorsovertheoceanisimportanttoNumericalWeatherPrediction(NWP) sincetheabilitytoproduceaforecastofthefuturestateoftheatmospheredependscriticallyon knowingthecurrentstateaccurately[HaltinerandWilliams,1980],particularlysincethesystem isnon-linear.However,theobservationnetworkovertheoceans(especiallyinthesouthernhemisphere)isverylimited[Daley,1991].Thusitishopedthattheglobalcoverageofoceanwind vectorsprovidedbysatellitebornescatterometerswillimprovetheaccuracyofnumericalweather forecastsbyprovidingbetterinitialconditions[HarlanandO'Brien,1986;Lorencetal.,1993]. ascyclones[DickinsonandBrown,1996]. 1994]andthepossibilityofstudying,athighresolution,interestingmeteorologicalfeaturessuch Thescatterometerdataalsooerstheabilitytoimprovewindclimatologiesovertheoceans[Levy,<br />
ERS-1<br />
785 km<br />
satellite track<br />
50km<br />
incidence angle<br />
mid<br />
beam<br />
TheERS-1satellitewaslaunchedinJuly1991bytheEuropeanSpaceAgency[Oler,1987].The Figure1:SchematicillustrationofthegeometryoftheERS-1satelliteshowingthefootprints<br />
azimuth angle<br />
bysmallripplesontheoceansurfaceofaround5cmwavelength,althoughthisdependsonthe on-boardmicrowaveradaroperatesat5.3GHzandmeasurestheamountofbackscattergenerated ofthethreeradarscatterometers.<br />
500 km aft<br />
beam<br />
incidenceangleoftheradarbeam.Measuredbackscatter<strong>from</strong>theoceansurfaceisgivenasthe NormalisedRadarCrossSection,andgenerallydenotedbyo,whichhasunitsofdecibels.A500 sampledacrosstheswathe,eachcellhavingdimensionsofroughly50by50km(seeFigure1). Thusthereissomeoverlapbetweencells.Also,eachcellissampled<strong>from</strong>threedierentdirections triplet,togetherwiththeincidenceangleofthemid-beam(whichvariesacrosstheswathe)canbe bythefore,midandaftbeamsrespectivelygivingatripletofobservations,(of;om;oa).Thiso kmwideswatheissweptbythesatellitealongthetrackofitspolarorbit,withnineteencells<br />
usedtodeterminetheaveragewindvectorwithinthecell[Oler,1994]. Anderson,1997]whereaphysicallybasedmapping<strong>from</strong>windvectorstooisformulated.In Manymethodstocomputewindvectors<strong>from</strong>scatterometerdataexist.Mosthaveconsidered modelbasedtechniques[Oler,1994;Wentz,1991;StoelenandAnderson,1992;Stoelenand aneuralnetworkbasedclassier,whichgaveprobabilitiesofthewinddirectionbeingineachof [Thiriaetal.,1993]themapping<strong>from</strong>otowindvectorswasmodelledusingsimulateddataand<br />
notstrictlysuchsincetheycantakenegativevaluesandarenotrequiredtosumtoone.Thewind Whiletheoutputsofthenetworksin[Thiriaetal.,1993]wereinterpretedasprobabilitiestheyare timetheworkwasundertaken. thirty-sixintervals.Simulateddatawasusedsincerealomeasurementswerenotavailableatthe<br />
of2361weights.Withatrainingsetofsizeapproximately6000observationsthereisconsiderable directionnetworkhas30inputs,2hiddenlayerseachof25unitsand36outputunitsgivingatotal dangerofover-tting[Bishop,1995].Despitethisthenetworksin[Thiriaetal.,1993]appear<br />
N<br />
fore<br />
beam
<strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData tohaveperformedverywellonthesimulateddata.Inthisstudyweuseaneuralnetworkto 3<br />
estimatethefulllocalconditionalprobabilitydensityofthewinddirectiongivenoinasimple andwell-foundedmanner[BishopandNabney,1996].<br />
Mucheorthasbeenputintounderstandingthetheoreticalrelationshipbetweenoandwind 2direction[Wentz,1991;StoelenandAnderson,1997].Thishasbeenbasedonstudiesofthe TheGeophysicalModel<br />
derived)andscatterometermeasurements[Oler,1994].Fromthesestudiesempiricalforward withastatisticalanalysisoftherelationbetweenwindvectors(bothbuoyobservedandNWP modelsbetweensingleo'sandrelativewinddirection(#)havebeenestablishedofthegeneral physicalprocessesthatgovernbackscattering<strong>from</strong>watersurfaces[Ebuchietal.,1993]together form wherethecoecientsarecomplicatedfunctionsofthescatterometerincidenceangle()andthe CMOD4[Oler,1994;StoelenandAnderson,1997].Therearethreeomeasurementsforeach windspeed(kuk).Themostwidelyused,andcurrentlyoperational,forwardmodelisknownas ob0+b1cos(#)+b2cos(2#) (1)<br />
cellandthesetogetherdeneaself-intersectingcone-likemanifoldin3dimensionalspace,which hasbeenshowntoapproximateaLissajouscurve[Thiriaetal.,1993].Formostotriples,which areobservedwithnoise,thereisambiguityovertheoptimaldirectiontoselect(seeFigure2).<br />
0<br />
180<br />
Figure2:Sketchofacrosssection(atconstantwindspeed)ofthe2Dmanifold(embeddedin pleobservationsareplottedasblackdots,withonelabelledtoshowthatthereisestimateoftheuncertaintyduetoinstrumenterrorandgeophysicalnoise.Exam- a3Dspace)ofthemapping<strong>from</strong>(o1;o2;o3)todirection().Thesolidlinegives theempiricalforwardmodel!o(e.g.CMOD4)whilethegreyareagivesan<br />
90<br />
generallymorethanonepossiblewinddirectionforagivenotriplet.<br />
270<br />
output(i.e.oasafunctionofwinddirection)isuni-valuedforagivensetofinputsbutthe inversemodel(i.e.winddirectionasafunctionofo)ismulti-valued.Itisknownthatthe Thisistypicalofmanyinverseproblemsinthegeophysicalsciences,wheretheforwardmodel relationbetweenwindspeedandoisuni-valued[Thiriaetal.,1993].Sincethewindspeedis largelyuncorrelatedwiththewinddirectionrelativetothesatelliteazimuth1angle,theproblem 1Theazimuthanglegivestheclockwiseangle<strong>from</strong>Northofthescatterometerbeamincidentonthecell.
4 <strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData<br />
ofmodellingwindvectorscanbesplitintomodellingthespeedanddirectionseparately.<br />
500<br />
400<br />
300<br />
200<br />
100<br />
0<br />
maximum speed 9.4 ms<br />
0 100 200 300 400 500<br />
Easting (km)<br />
−1<br />
300<br />
(a)<br />
200<br />
100<br />
0<br />
maximum speed 15.0 ms<br />
0 100 200 300 400 500<br />
Easting (km)<br />
−1<br />
Figure3:Twoscenesshowingthetargetwindvectorsfor(a)agradientinwindspeedand (b)<br />
usingtheCMOD4forwardmodelandminimisingsomecostfunction(whichistypicallyamean Operationally,theproblemofobtaininglocalwinddirections<strong>from</strong>scatterometerdataisresolved squareerror)betweentheobservedotripletsandthemanifolddenedbyCMOD4.Ingeneral directionand(b)acycloniccirculation.<br />
uptofourvalidsolutionsareobtained(althoughthereareoftentwodominantmodeswithapproximately180degreeambiguity|thetrueandaliassolutions).Disambiguationmethods[Chelton todecidewhichlocaldirectionistobeselected,oftenbasedonthespatialcorrelationpresentin etal.,1989;Schultz,1990;Shaeretal.,1991](suchassmoothinglters)arethenappliedglobally addressedinfuturework2.AnadvantageofourprobabilisticmodelsisthatthisallowsBayesian windelds(seeexamplesinFigure3).Ambiguityremovalisnotdiscussedinthispaperbutwillbe ofwindspeedanddirectiongiventhelocaloobservation. methodstobeappliedtothedisambiguationproblem.Hereweconsideronlythelocalprediction<br />
3Manyapplicationsofneuralnetworkscanbeformulatedintermsofamultivariatenon-linearmap ping<strong>from</strong>aninputvectorxtoatargetvectort.Aconventionalneuralnetworkapproach,based <strong>Neural</strong><strong>Network</strong>sforModellingScatterometerData<br />
onaleastsquareserrorfunction,forexample,leadstoanetworkmappingwhichapproximates valued,suchaswinddirectioninthisapplication,thisapproachbreaksdown,sincetheaverageoftheregression(i.e.theconditionalaverage)oftgivenx.However,formappingswhicharemulti- twosolutionsisnotnecessarilyavalidsolution. Thisproblemcanberesolvedbyrecognisingthattheconditionalmeanisjustoneaspectofa morecompletedescriptionoftherelationshipbetweeninputandtarget,obtainedbyestimating thefullconditionalprobabilitydensityoftconditionedonx,writtenasp(tjx).Theleastsquares byaGaussiandistributionwhichissphericallysymmetricint-spaceandwhichhasanx-dependent approachthencorrespondstomaximumlikelihoodforthespecialcaseinwhichp(tjx)ismodelled<br />
http://www.ncrg.aston.ac.uk/Projects/NEUROSAT/.<br />
meanandconstantvariance.Themapping<strong>from</strong>otodirectionwilltypicallybemulti-valuedand thusourmodelofp(#jo)cannotbemodelledbyauni-modaldistribution;insteadweshowhow 2More details of the project this work formspart of can be found at<br />
Northing (km)<br />
Northing (km)<br />
500<br />
400
<strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData mixturesofuni-modaldistributionscanbeused.Thewindspeedmapping,ontheotherhand,is 5<br />
uni-valuedandcanbesensiblymodelledusingaregressionapproach,asoutlinedbelow.Thuswe addresspredictionin(kuk;#)spaceratherthanusingtheCartesianvectorcomponents. 3.1<strong>Neural</strong><strong>Network</strong>sforModelling<strong>Wind</strong>Speed Ourmodelforpredictingwindspeedisafullyconnectedtwolayermulti-layerperceptronwith sigmoidalactivationfunctionsinthehiddenlayerandexponentialunitsintheoutputlayer,to ensurethatonlypositivespeedsaregenerated:<br />
withtheoutputlayer: zj=g dXi=0w(1) jixi!; (2)<br />
wherekupkisthepredictedwindspeed,Histhenumberofhidden(sigmoidal)unitszj,disthe numberofinputsxi(forournetworksthiswasfour-theotripletandthemid-beamincidence kupk=exp0@HXj=0w(2) jzj1A; (3)<br />
angle).Thew'srepresenttheweightsofthenetworkwithw(1) unittotheoutput(windspeed).w(1) theithinputtothejthhiddenunitandw(2) unitsrespectively.Thefunctiongisthesigmoidfunction: j0andw(2) jbeingthesecondlayerweight<strong>from</strong>thejthhidden 0arethebiasparametersforthehiddenandoutput jibeingtherstlayerweights<strong>from</strong><br />
Thenetworkwastrainedusingasumofsquareserrorfunction: g(a)= 1+exp(?a): 1 (4)<br />
thekthexampleforthetrainingsetandkukkisthekthtargetvalue;thatistheobservedspeed wherenisthenumberofobservationsinthetrainingset,kupkkistheoutputofthenetworkfor Ekupk=12nXk=1(kupkk?kukk)2 (5)<br />
forthekthexampleforthetrainingset.Back-propagation(todeterminethegradientoftheerror functionwithrespecttothenetworkweights)togetherwithaconjugategradientoptimisation algorithmwasappliedtodeterminetheoptimalweightsinthenetworks.Only500iterationsof thisalgorithmwererequiredforconvergenceforallnumbersofhiddenunitsinvestigated.Early stoppingusingindependenttrainingandvalidationsets[Bishop,1995,Section9.2.4]reducedthe quadraticregressionmodelsoftheform possibilityofover-ttinganddierentnumbersofhiddenunitswereinvestigated. Inordertoassessthedegreeofnon-linearityinthewindspeedretrievalproblem,linearand<br />
datasets.Theparameterswerecomputedusingstandardleastsquaresestimationonthetraining wherePistheorderofthepolynomial(1or2)anddisasbefore,werealsotestedonthesame kupk=exp0@w0+PXj=1dXi=1wijxji1A; (6)<br />
set[Pressetal.,1992].
6 3.2<strong>Neural</strong><strong>Network</strong>sforModelling<strong>Wind</strong>Direction <strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData<br />
Whenmodellingwinddirectionsomecaremustbetakensincenotonlyisthetargetvariable multi-valued,itisalsoperiodic.Thissectiondescribesonemethodofestimatingtheconditional details. (probability)densityofperiodicvariables:seealso[BishopandNabney,1996]forcomputational<br />
Acommonlyusedtechniqueforunconditionaldensityestimationisbasedonmixturemodelsof 3.2.1DensityEstimationforPeriodicVariables theform<br />
whereiarethemixingcoecients,andthelcomponentfunctions,orkernels,i(t),aretypically chosentobeGaussians[Titteringtonetal.,1985;McLachlanandBasford,1988].Inordertoturn p(t)=lXi=1ii(t); (7)<br />
parametersinthecomponentdensities,aresettobefunctionsoftheinputvectorx: thisintoamodelforconditionaldensityestimation,themixingcoecients,aswellasanyadaptive<br />
Thesefunctionsarelikelytobenon-linear,sowesetthemixingcoecientsandkernelparameters <strong>from</strong>theoutputsofaneuralnetworkwhichtakesxasinput.Thisunderliesthe`mixtureof p(tjx)=lXi=1i(x)i(tjx): (8)<br />
experts'model[Jacobsetal.,1991]andhasalsobeenconsideredbyanumberofotherauthors [Bishop,1994;Liu,1994]. Inthissectiontwomethodsformodellingtheconditionaldensityp(#jx)ofaperiodicvariable# conditionedonaninputvectorxarereviewed.Bothmethodsusethesamekernelfunctions(and outputerrorfunction)butallowdierentsetsofparameterstobevaried. 3.2.2CircularNormalDensities Byusingamixtureofkernelfunctionsin(8)whichareperiodicthemselvestheoverallconditional densityfunctionwillbeperiodic.Thekernelfunctionofthewinddirection#isgivenby:<br />
coecientisexpressedintermsofthezerothordermodiedBesselfunctionoftherstkind, whichisknownasacircularnormalorvonMisesdistribution[Mardia,1972].Thenormalisation i(#)= 2I0(mi)expfmicos(#? 1 i)g; (9)<br />
I0(mi),andtheparametermiisanalogoustotheinversevarianceparameterinaconventional Amulti-layerperceptron,withasinglehiddenlayerofsigmoidalunits(2)andlinearoutputunits, normaldistribution.Theparameter isusedtosettheparametersinthemixturemodel(8)and(9).Thelinearoutputsaregivenby: iisthemeanofthedensityfunction.<br />
zok=HXj=0w(2) kjzj; (10)
<strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData whereHisthenumberofhiddenunitsandw(2) kjweight<strong>from</strong>thejthhiddenunit(zj)tothe 7<br />
kthlinearoutput.Wedividethenetworkoutputsintothreeclasses,correspondingtomixing Inordertoensurethatthemixturemodelin(8)isaprobabilitydensityfunction,itissucient thatthemixingcoecientsi(x)satisfytheconstraints: coecients,meansandvariancesandapplyasuitabletransformtoeachclassofoutput.<br />
=1i(x)=1; lXi outputsbyanormalisedexponential,orsoftmaxfunction[Jacobsetal.,1991]: forallx.Thiscanbeachievedbychoosingtheitoberelatedtothecorrespondingnetwork 0 i(x)1; (11)<br />
i= Plj=1exp?zj; exp(zi)<br />
themixingcoecients).Thecentres wherezjrepresentthecorrespondingnetworkoutputs(i.e.thecomponentofzokwhichrepresents i(x)ofthekernelfunctionsarerepresenteddirectlyby (12)<br />
thenetworkoutputssincethesemaytakeanyvalueoverthereals.Thisisalsomotivatedby thecorrespondingchoiceofanuninformativeBayesianprior,assumingthattherelevantnetwork<br />
solution.Again,itcanbemotivatedbytheconceptofanuninformativepriorintheBayesian varianceparametersmi(x)ofthekernelfunctionsarescaleparametersandsoitisconvenientto outputshaveuniformprobabilitydistributions[Jacobsetal.,1991;Berger,1985].Theinverse representthemintermsoftheexponentialsofthecorrespondingnetworkoutputs.Thisensures thatmi(x)>0anddiscouragesmi(x)<strong>from</strong>tendingto0,whichcorrespondstoapathological framework.<br />
Theothertechniqueusedinthispaperinvolvesaconditionaldensitymodelasin(8)consistingof axedsetofperiodickernels,againgivenbycircularnormalfunctionsasin(9).Inthiscasethe 3.2.3ExpansioninFixedKernels<br />
mixingcoecientsalonearedeterminedbytheoutputsofaneuralnetwork(throughasoftmax uniformdistributionofcentres,andsetmi=mforeachkernel,wherethevalueformwas activationfunction(12))andthecentres chosentogivemoderateoverlapbetweenthekernelfunctions.Fixedkernelsareonlyappropriate fortargets<strong>from</strong>lowdimensionalspacessincethenumberneededgrowsexponentiallywiththe dimensionofthetargetspace. iandscaleparametersmiarexed.Weselecteda<br />
Theotripletogetherwiththe(mid-beam)incidenceangleandwindspeedpredictedby(3)were 3.2.4ComputationalDetails usedasinputstothemixturedensitynetwork(i.e.x=(o;;kupk)).Theuseofincidenceangle asanadditionalinputtoourneuralnetworkmakesourmodelmoreexiblethanthosein[Thiriaet thattherelationshipbetweendirectionandoisdependentonthewindspeed. onlyeveryothercell).<strong>Wind</strong>speedwasincludedasaninputsince[Thiriaetal.,1993]suggested al.,1993]whereaseparatenetworkwastrainedforeachofthe10incidenceangles(theyconsidered<br />
ofEwithrespecttothenetworkweightscanbecomputedusingtherulesofcalculus[Bishopand<br />
Forbothmodelstheadaptiveparametersofthemodel(theweightsandbiasesinthenetwork)are optimisedbymaximisingthelikelihoodofthedatagiventhemodel.Inpracticeitisconvenientto minimiseanerrorfunctionEgivenbythenegativelogarithmofthelikelihoodfunction.Derivatives
8 Nabney,1996],andthesederivativescanthenbeusedwithstandardoptimisationproceduresto <strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData<br />
networkweightssothatthecentresareapproximatelyevenlyspaced,thescaleparametersarelarge goodinitialisationofthemodelparameterstoavoidbadlocalminima.Inthiscaseweinitialisethe enoughtoensureoverlappingofthekernelfunctionsandthemixingcoecientsareapproximately equal.Theseare,however,setwithasmallrandomcomponentsothatwecaninvestigatethe ndaminimumoftheerrorfunction.Insuchnon-linearproblemscaremustbetakentoensure<br />
stopping[Bishop,1995]wasusedtoensurereasonablegeneralisationperformance.Thenetwork eectofdierentinitialisations. Aconjugategradientalgorithmwasusedtominimisethemixturedensityerrorfunction.Early thatwasusedonthetestsetwastheonewiththelowestvalidationseterrorEv.Generally Normal(CN)network,andthiswasalwaysattainedwithin2000iterations.Whentrainingthe afteraround1200iterationstheearlystoppingrulehadselectedthebestfullyadaptiveCircular FixedKernel(FK)networksconvergencewasmuchquicker,usuallyoccuringwithin250iterations. Severalnetworkarchitectureswereinvestigatedbyvaryingthenumberofhiddenunitsandthe numberofcircularnormalfunctionsused.Whenusingxedkernelsthescaleparametermwas<br />
thecommitteewereweightedequallysinceallhadsimilaraccuracies. combinedthepredictiveconditionaldensitiesp(#jx)<strong>from</strong>severalmodels.Thenetworksforming Oncethenetworksweretrained,acommitteeofnetworks[Bishop,1995]wasconstructedwhich alsovaried.<br />
proposedbyDavidOleroftheUKMeteorologicalOce[StoelenandAnderson,1997].The 4InordertocomparedierentmodelsaFigureofMerit(FoM)evaluationfunctionhasbeen EvaluationFunction<br />
speedrange.AFoMofgreaterthanoneindicatesthetransferfunctionisperformingtowithin thesespecications,althoughtheexactformisratherad-hoc.TheFoMdetailscanbefound forwindspeedand20oforwinddirection.TheFoMiscomputedoverthe4?24ms?1wind FoMreectstheextenttowhichthetransfermodelmeetsthedesignspecicationsof2ms?1<br />
inAppendixI.TheFoMisconsideredinbothweightedandunweightedforms,whichtakeinto accounttheperformaceofthealgorithmatdierentwindspeeds.Ifwewishtoperformwellon thewindspeedclass)orcarefullyselectthetrainingset.Inthisstudywechosethelatteroption. thesumoftheweightedandunweightedevaluationfunctionsthenwemusteitherexplicitlyadapt thecostfunctioninthenetworktraining(byweightingtheerrorfunctionbyafactordependingon<br />
theobserveddirectionthenthethirdmostlikelydirectionischosenandsoonuntiltherstfour observeddirectionthesecondmostlikelydirectionischosen.Ifthisisstillmorethan90o<strong>from</strong> ThewinddirectionusedwhencomputingtheFoMwaschosenusingaverysimpledisambiguation<br />
becalculatedintheabsenceofamoresophisticatedambiguityremovalalgorithm.Thealgorithm modeshavebeenconsidered.Thismustbedonetoallowasensiblevalueofthedirectionerrorsto algorithm.Initiallythemostlikelydirectionisselected.Ifthisismorethan90o<strong>from</strong>the<br />
usedtocomparetheperformanceof`localmodels'. usedhereisnotapplicableinpracticesinceitrequiresknowledgeofthetargetvalues,andissimply<br />
5ThedatausedinthisstudywascompiledbytheEuropeanSpaceAgencyincollaborationwith theUKMeteorologicalOce.Thedatabaseconsistedof115scenes,eachofwhichcontains19by<br />
Data
<strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData 19cells,orobservationlocations,correspondingtoasquareofarearoughly500by500km.Two 9<br />
examplescanbeseeninFigure3.Thesceneswereclassiedinto6cases: lowwindspeeds(10scenes)<br />
cycloniccirculations(10scenes) gradientsinspeedordirection(59scenes) homogeneouscases(15scenes)<br />
anti-cycloniccirculations(11scenes)<br />
Thisclassicationwasmadebymeteorologists.Eachofthe39,611observationsinthisdataset containedinformationonitslocation,theotriple,incidenceandazimuthangles,windspeed frontsandother`dicult'cases(10scenes)<br />
andwinddirection.Inthisworkallwinddirectionswerecomputedrelativetotheazimuthangle, sothewinddirectioninthetrainingdatadoesnotgivetheabsolutedirection,butratherthe relativedirection.Thewindspeedanddirectionwereobtained<strong>from</strong>theUKMeteorologicalOce UniedModel3,whichhasahorizontalresolutionofapproximately150km[MiltonandWilson, 1996].Thusthetargets(i.e.windspeedanddirection)areinterpolated<strong>from</strong>amodeldesignedto representthelargescaledynamicsoftheatmosphere[HaltinerandWilliams,1980].Thismeans thereislikelytobeconsiderablesmoothingofthetargetvalues. WeuseNWPmodelwindeldssincethesearethebestestimatesofthetruewindsavailableover NWPmodelderivedestimatesofwindspeedanddirectionarebasedonassimilateddatawhich theoceansurfacethatarecollocatedwiththescatterometerobservations.Caremustbetaken sincethereisadangerinusingonemodeltodenethetargetsforanothermodel.Howeverthe combinesthoseindepedentobservationsavailable<strong>from</strong>bouysandshipswithagoodforecast<strong>from</strong> thepreviousstateoftheatmosphere,andrepresentsthebestavailableestimateofthetruewinds. Duetothesparsityofobservationsovertheoceanstheforecaststatedominatestheanalysisand thelocationandintensityoffeaturesinthewindeldsmaybeinerror,thusthequalityofthe targetswasimprovedbymatchingthepositionoflowpressurecentresintheforecastwindswith thoseobservedbythescatterometerbylinearlytranslatingthemodelwindelds. vations,sincethewithin-scenespatialcorrelationsbetweenbothwindspeedandwinddirectionDespitehavingnearly40,000observations,thereareinrealityfarfewertrulyindependentobser- willbeveryhigh.Thisimpliesthatcaremustbetakenwhenselectingdatatotrain,validateand testtheneuralnetworkmodels.Thelocalmodelswearetrainingconsideronlytheinformation <strong>from</strong>withintherelevantcelltoinferwindspeedanddirection.Thustoretrieveallspeedsand directionswellwerequireatrainingsetwithallpossiblecombinationsofwindspeed,directionand<br />
Figure4showsthedistributionofthetargetvaluesinthese102scenes.<strong>Wind</strong>speedhasadistribu- 102sceneswereusedforparameterestimation. sceneswereselected<strong>from</strong>thewholedatasetandremoved<strong>from</strong>furtheranalysis.Theremaining beamincidenceanglerepresented.Inordertoretainsometotallyindependentdatafortesting13<br />
tionstronglyskewedtowardslowerspeeds,whichreectsthedistributionofsurfacewindspeedsin theatmosphere.Wewishtotrainournetworkstolearnthetransferfunctionatallspeedrangesto minimisetheFoMevaluationfunction.Thuswhenselectingasubsetof3,000observationstotrain thenetworks,1,500observationswereselectedtogiveasuniformadistributionofwindspeedas possible.Afurther1,500randomlychosenobservationswerealsoselected.Thisimpliesthatequal weightisgiventotheweightedandunweightedFoM.Thevalidationset,whichwasusedinthe 3Seehttp://www.meto.govt.uk/sec5/NWP/NWP.html.
10 <strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData<br />
4500<br />
4000<br />
3500<br />
3000<br />
2500<br />
2000<br />
1500<br />
1000<br />
500<br />
0<br />
0 5 10 15 20 25 30<br />
<strong>Wind</strong> Speed (ms −1 1500<br />
(a) (b)<br />
1000<br />
500<br />
Figure4:Histogramsshowingthedistributionof(a)windspeedand(b)direction.<br />
0<br />
0 50 100 150 200 250 300 350<br />
)<br />
<strong>Wind</strong> Direction (degrees)<br />
earlystoppingprocedure,wasselectedinasimilarmanner.Finallyatestsetof3,000observations waschosenrandomly<strong>from</strong>thetestscenes.Allvariables(exceptwindspeedanddirection)were linearlytransformedtohavezeromeanandunitvariance,usingthemeanandstandarddeviation derived<strong>from</strong>thedirectiontrainingdataset.Allresultsinthispaperrefertothetestset. 66.1<strong>Wind</strong>Speed Results<br />
Frequency<br />
Predicted <strong>Wind</strong> Speed (ms −1 )<br />
25<br />
20<br />
15<br />
10<br />
5<br />
0<br />
0 5 10 15 20 25<br />
Observed <strong>Wind</strong> Speed (ms −1 usingstandardregressiontechniques.Figure5showsascatter-plotoftheresultsforthe4hidden<br />
Therelationshipbetweenwindspeedandoisthelesschallengingproblemandcanbeapproached Figure5:Scatter-plotofpredictedversusobservedwindspeedsusingthe4hiddenunitmultilayerperceptron. )<br />
Frequency<br />
2500<br />
2000
<strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData unitnetwork,demonstratingthatareasonableapproximationismadeatallwindspeeds,although 11<br />
therearesomelargeresiduals.Thewindspeedresultsarealsocomputedfordierentwindspeed bins,assuggestedinthesectiononevaluationfunctions. Table1:Resultsforthe4hiddenunitmulti-layerperceptron,binnedbywindspeedanda linearregressionmodel.Theregressionmodelincludedtermsuptoquadraticinthe variablesbutnointeractionterms.Allguresgiveninms?1. SpeedRange 8{12 4{8 20 -0.742.13 -0.302.17 -0.941.23 2.25 2.19 1.55 324 126<br />
aStandardDeviationoptimalregressionmodel-0.141.85<br />
wholetestset 0.231.80 1.85 1.81 3000<br />
11<br />
bRootMeanSquareError forspeedathigherandlowerwindspeeds(shownespeciallyinthelargebiases).Thisisafeature FromTable1itisclearthatthenetworkishavingsomedicultylearningthetransferfunction cNumberofobservations<br />
commontoallreportedtransfermodels[Oler,1994;Wismann,1992]andtheregressionmodel. Therootmeansquareerrorof1.81ms?1forthefulltestsetiswithinthedesignspecicationof theinstrumentof2ms?1. 6.2<strong>Wind</strong>Direction<br />
Conditional Probability Density<br />
1.5<br />
1<br />
FK 48 36 0.6<br />
FK 24 18 0.7<br />
CN 50 4<br />
CN 12 6<br />
Committee<br />
←true direction ←alias direction<br />
1<br />
0.8<br />
Figure6:Conditionaldensityfunctionsfor2cellsshowingtheresultsof2circularnormal,2<br />
0.5<br />
0.6<br />
0.4<br />
0.2<br />
Figure6illustratesthatthedierenttechniquesofusingadaptivecircularnormalsandexpansion incorrectby180degrees)targetsareshown. xedkernelandthecommitteeofthese4models.Boththetrueandaliased(i.e.<br />
0<br />
0<br />
−3 −2 −1 0 1 2 3<br />
−3 −2 −1 0 1 2 3<br />
Direction (radians)<br />
Direction (radians)<br />
inxedkernelsproducessimilarconditionaldensitiesforthewinddirectiongiventhescatterometer inputs.Thereishoweversomevariabilityintheresults,suggestingthatacommitteeofnetworks mightimproveperformance.Inbothcasesthereisgoodagreementbetweentheobserveddirection<br />
Conditional Probability Density<br />
2<br />
1.8<br />
1.6<br />
1.4<br />
1.2<br />
FK 48 36 0.6<br />
FK 24 18 0.7<br />
CN 50 4<br />
CN 12 6<br />
Committee<br />
←alias direction<br />
←true direction
12 <strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData<br />
Predicted Direction (radians)<br />
3<br />
2<br />
1<br />
1<br />
0<br />
(a)<br />
0<br />
−1<br />
−1<br />
−2<br />
−2<br />
Figure7:Scatter-plotsofobservedversuspredicteddirectionfor(a)themostlikelyand(b) (b)<br />
thesecondmostlikelysolutions.Thelinesshowperfecttand180oaliassolutions.<br />
−3<br />
−3<br />
−3 −2 −1 0 1 2 3<br />
−3 −2 −1 0 1 2 3<br />
Observed Direction (radians)<br />
Observed Direction (radians)<br />
solutions;thetrueoneandits180degreealias).AlsomarkedonFigure7isthelineofcorrect choosingthemostlikelyandsecondmostlikelyvectors(sincegenerallytherearetwodominant solutions(observed=predicted)inthecentre,andthetwolines(observed=predicted180 andthepredictedconditionaldensities.Figure7showstheresultsofthecommitteeofnetworks, degrees)thatarethealiassolutions.Thedataclustersaroundtheselinesinbothgures,although thereisconsiderablescatter.<br />
Table2:Resultsofthenetworktechniquesappliedtothedirectiondata.Columnsgivethe<br />
<strong>Network</strong>Conguration/Technique1solution2solutions4solutions percentageofobservationswithin20oofthetargetsolutionconsideringthemost likely,rstandsecondmostlikelyandtherstfourmostlikelydirections.Foreach examplethebestttingsolutionischosen. FK24180.7a FK48360.6 CN126b 51.7 53.0 47.2 70.9 72.0 70.8 86.5 79.0<br />
aFK=xedkernel,numberofhiddenunits,numberofxedkernels,scaleofkernelsrelativetointer-kernel 4netcommittee CN504 45.1 53.8 72.9 74.2 78.7 84.0 77.5<br />
spacing. bCN=circularnormals,numberofhiddenunits,numberofadaptivekernels.<br />
onlyoneortwosolutions,howeverwhenconsideringthefourmostlikelysolutionssomeofthe Table2showsselectedresultsfortwoofthebetterxedkernelandcircularnormalapproachesas than70%ofthetime.Thecommitteeofnetworksoutperformalltheirmemberswhenconsidering wellasthecommitteeresults.Byconsideringbothrstandsecondsolutionsandpickingthebetter one,usingthecircularnormaltechniqueweobtainthecorrectsolutionwithin20degreesmore xedkernelresultsaremarginallybetter.<br />
Predicted Direction (radians)<br />
3<br />
2
<strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData 13<br />
Table3:Comparisonofresultsusingdierentalgorithms(resultscomputedinspeedrangethe 4{24ms?1).NotethatthenalCMOD4resultusesacompletelydierentdataset,<br />
<strong>Neural</strong>networks-thisstudy andthattheotherstudiesusedadierentambiguityremovalprocedurethanwas<br />
CMOD4-samedata usedinthisstudy.Allunitsms?1(speed)ando(direction).<br />
Subsetofdata[26] MethodUsed SpeedBiasSpeedSDaDir.BiasDir.SD -0.01 -0.44 0.3 1.73 1.8 2 -1.37 0.73 { 23.05 {<br />
aStandardDeviation CMOD4[11] CMOD4[8] 0.06 0.1 1.65 1.9 0.76 -1.6 16.69 17.0 20<br />
7TheequivalentresultsfortheoperationalCMOD4algorithmareonlyavailableforan(unknown) DiscussionandConclusions<br />
subsetofthedatasetweused,excludingthosecaseswithwindspeeds
<strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData Harlan,J.andJ.J.O'Brien1986.AssimilationofScatterometer<strong>Wind</strong>sintoSurfacePressure 15<br />
Jacobs,R.A.,M.I.Jordan,S.J.Nowlan,andG.E.Hinton1991.Adaptivemixturesoflocal Levy,G.1994.Southern-HemisphereLow-Level<strong>Wind</strong>CirculationStatistics<strong>from</strong>theSeaSat FieldsUsingaVariationalMethod.JournalofGeophysicalResearch91,7816{7836. experts.<strong>Neural</strong>Computation3,79{87.<br />
Liu,Y.1994.Robustneuralnetworkparameterestimationandmodelselectionforregression. Scatterometer.AnnalesGeophysicae-Atmospheres,HydroshperesandSpaceSciences12, 65{79.<br />
Lorenc,A.C.,R.S.Bell,S.J.Foreman,C.D.Hall,D.L.Harrison,M.W.Holt,D.Oler, InAdvancesin<strong>Neural</strong>InformationProcessingSystems,Volume6,pp.192{199.Morgan Kaufmann.<br />
Mardia,K.V.1972.StatisticsofDirectionalData.London:AcademicPress. McLachlan,G.J.andK.E.Basford1988.Mixturemodels:InferenceandApplicationstoClus- andS.G.Smith1993.TheUseofERS-1ProductsinOperationalMeteorology.Advancesin SpaceResearch13,19{27.<br />
Milton,S.F.andC.A.Wilson1996.Theimpactofparameterizedsubgrid-scaleorographic forcingonsystematicerrorsinaglobalNWPmodel.MonthlyWeatherReview124,2023{ tering.NewYork:MarcelDekker.Oler,D.1994.TheCalibrationofERS-1<strong>Satellite</strong>Scatterometer<strong>Wind</strong>s.JournalofAtmoOler,D.1987.<strong>Wind</strong>Measurements<strong>from</strong>theEarthRemote-sensing<strong>Satellite</strong>(ERS-1).MeteorologicalMagazine116,279{285.<br />
2045.<br />
Schultz,H.1990.ACircularMedianFilterApproachforResolvingDirectionalAmbiguitiesin Press,W.H.,S.A.Teukolsky,W.T.Vettering,andB.P.Flannery1992.NumericalRecipesin sphericandOceanicTechnology11,1002{1017. C(2ndEditioned.).Cambridge,UK:CambridgeUniversityPress.<br />
Shaer,S.J.,R.S.Dunbar,S.V.Hsiao,andD.G.Long1991.AMedian-Filter-<strong>Based</strong>Ambiguity <strong>Wind</strong>FieldsRetrieved<strong>from</strong>SpaceborneScatterometerData.JournalofGeophysicalResearch95,5291{5303.Stoelen,A.andD.Anderson1997.ScatterometerDataInterpretation:EstimationandValidationoftheTransferFunctionCMOD4.JournalofGeophysicalResearch102,5767{5780. 167{174. RemovalAlgorithmforNSCAT.IEEETransactionsonGeoscienceandRemoteSensing29,<br />
Thiria,S.,C.Mejia,F.Badran,andM.Crepon1993.A<strong>Neural</strong><strong>Network</strong>ApproachforModeling Stoelen,A.andD.L.T.Anderson1992.ERS-1ScatterometerDataCharacteristicsand<strong>Wind</strong> theServiceoftheEnvironment,pp.41{47.Cannes,France,ESA. <strong>Retrieval</strong>Skill.InB.Kaldeich(Ed.),ProceedingsoftheFirstERS-1Symposium-Spaceat<br />
Titterington,D.M.,A.F.M.Smith,andU.E.Makov1985.StatisticalAnalysisofFinite NonlinearTransferFunctions:Applicationfor<strong>Wind</strong><strong>Retrieval</strong><strong>from</strong>SpaceborneScatterometerData.JournalofGeophysicalResearch98,22827{22841. Wismann,V.1992.AC-Band<strong>Wind</strong>ScatterometerModelDerived<strong>from</strong>theDataObtained Wentz,F.J.1991.ASimplied<strong>Vector</strong>Algorithmfor<strong>Satellite</strong>Scatterometers.JournalofAtmosphericandOceanicTechnology8,697{704. MixtureDistributions.Chichester:JohnWiley.<br />
theFirstERS-1Symposium-SpaceattheServiceoftheEnvironment,pp.55{59.Cannes, DuringtheERS-1Calibration/ValidationCampaign.InB.Kaldeich(Ed.),Proceedingsof France,ESA.
16 AppendixI<strong>Neural</strong><strong>Network</strong><strong>Based</strong><strong>Wind</strong><strong>Vector</strong><strong>Retrieval</strong><strong>from</strong><strong>Satellite</strong>ScatterometerData<br />
InordertocomparedierentmodelsaFigureofMerit`evaluationfunction'hasbeenproposedby takestheform: DavidOleroftheUKMeteorologicalOce(personalcommunication).Thisevaluationfunction<br />
givesthewindspeed,DthewinddirectionandVthewindvector.Thisreectstheextentto whereF1=40=(Ubias+10Usd+Dbias+Dsd);F2=(2=Urms+20=Drms)=2;F3=4=Vrms;U FoM=(F1+F2+F3) 3 (13)<br />
whichthemodelmeetstheinstrumentspecicationsof2ms?1and20o.AFoMofgreater thanoneindicatesthetransferfunctionisperformingtowithinthesespecications,althoughthis isaratherad-hocmeasure.Thebiasisgivenby:<br />
wheretheresidualwindspeedUres=Upred?Uobs,thepredictedspeed(Upred)coming<strong>from</strong>the Ubias=1nnXi=1Ures(i) (14)<br />
neuralnetwork,theobservedspeed(Uobs)<strong>from</strong>thenumericalweatherpredictionmodelandnis thenumberofobservations.Thestandarddeviationoftheresidualsisgivenby:<br />
SimilarlytherootmeansquareerrorUrmsisgivenby: Usd=vut1nnXi=1(Ures(i))2?(Ubias)2 (15)<br />
ThevectorresidualVresisgivenby: Urms=vut1nnXi=1(Ures(i))2 (16)<br />
ThustheFoMiscomputedbybinningthecasesin5windspeedclasses;4?8ms?1,8?12ms?1, Thewindspeedisknowntoaecttheabilityoftheon-boardinstrumentstoresolvewinddirection. Vres=qU2obs+U2pred?2UobsUpredcos(Dres) (17)<br />
12?16ms?1,16?20ms?1and>20ms?1.Thebias,standarddeviationandrootmeansquare (bythenumberofobservationsineachbin)andunweightedmeansofthebinnedstatistics.The error(oftheresiduals)arecomputedforeachbinandanalFoMiscomputed<strong>from</strong>bothweighted unweightedmeangivesmuchlargerimportancetoperformanceathigherwindspeeds,whichare weightedmeantakesintoaccountthedistributionofwindspeedvaluesintheatmosphere.The arguablythecasesofgreatestinteresttoatmosphericscientists.
Change language