Electrical thunderstorm nowcasting using lightning data ... - Vaisala

200619thInternational Lightning Detection Conference24-25 April • Tucson, Arizona, USA1st International Lightning Meteorology Conference26-27 April • Tucson, Arizona, USAELECTRICAL THUNDERSTORM NOWCASTING USING LIGHTNING DATA MININGCarlos Alberto Vasconcellos, Cesar Beneti, Fábio Sato, Luciane Cristina PinheiroSIMEPAR Technological Institute - Curitiba – BrazilClaudio Luiz CurottoParana Federal University/UFPR – Curitiba – Brazil1. INTRODUCTIONThis paper presents a study developed atSIMEPAR (Paraná state weather service)using lightning data for electrical thunderstormnowcasting.Thunderstorm electrical data collected atSIMEPAR such as lightning location, time ofoccurrence, current intensity, polarity, etc, isstored in real-time in a relational database. Asa first step of this study Microsoft BusinessIntelligence bundled with SQL Server 2005Beta was used to access some of these dataand create lightning clusters (Msdn, 2005)representing electrical thunderstorms. Theclusters were continuously monitored topredict electrical thunderstorm displacementand evolution.Work is undergone to assess suitabilityand reliability to the process. Algorithmparameters fitting and cases studies are underdevelopment and further work will be doneusing Weka custering classes (Witten andFrank, 2005). Once approved, themethodology will be integrated to SisRaios - aJava lightning data visualization, analysis andthunderstorm monitoring and forecasting tool(Beneti et al., 2002).The aim of clustering (Fayyad et al., 1996)lightning data is to track the clustersdisplacement allowing thus electricalthunderstorm nowcasting by extrapolation.First analysis was made with a sample of2.000 thunderstorm electrical discharges atParaná state (south of Brazil) with 30 minutestime window. Microsoft Clustering algorithmwas employed using SQL Server 2005 Beta.Despite the geographical nature of theevent which suggests Euclidean distancemeasurements K-Means was adopted asclustering method. Scalable EM (Expectation-Maximization) was also studied because itbest handles large data sets (Bradley et al.,1998). Non scalable K-Means brought betterresults in position sense.Figure 1 shows that clusters obtained atthis analysis configuration are visually quiterepresentative to the lightning positions. Next30 minutes increase in time window showedhowever a poor relationship between clustersfrom first and second time windows.It is expected that this computational toolenhanced by the data mining study will aidmeteorologists and power companies tomonitor electrical thunderstorms, supplyinginformation for starting up maintenance teams,as well as providing a better thunderstormwarn to civil in general and improvingSIMEPAR's nowcasting capabilities.2. DISCUSSION AND RESULTSFigure 1. 30 minutes time window clusters1

In order to be able to track the clustersdisplacements, a second approach was toreduce the time window to 15 minutes. It wasalso changed the cluster count parameter fromthe default value (10) to the best count foreach case (selected automatically by thealgorithm). Figure 2 shows na example with 3clusters generated with the changes describedabove (Time step 1).Further 15 minutes were increased and insome cases the clusters tracks were lost, asshown in Figure 4 (Only 2 clusters plotted).Figure 4. 15 minutes time window clusters.Time step 3.Figure 2. 15 minutes time window clusters.Time step 1.The calculated locations in the next 15minutes step clusters (shown in Figure 2)corresponded to the actual lightning activitytrack (except for the cluster 3, which shouldmove to the east but moved to the west). Theupdated clusters positions are depicted in theFigure 3 below.These clusters track losses can be due tothe uncoupled phenomenon nature. Othervariables (such as orography, cold fronts,radar and satellite data) should be analyzedintegrated (Beneti et al., 2004).3. CONCLUSIONSThe clustering analysis using MicrosoftClustering Algorithm lead to good results inwell behaved weather conditions (nonconvective thunderstorms) considering onlymatematical data issues. Non scaling K-Means, with automatic cluster counting in 15minutes time window were the bestconfiguration scheme used.Other algorithms and parametersadjustments will be evaluated, including WekaJava classes in order to add Data Miningcapability at SIMEPAR's visualization andanalysis tool (SisRaios).4. REFERENCESBeneti, C., Vasconcellos, C. and Sato, F.2002: A Web Based Application forLightining Analysis and thunderstormForecasting. ILDC. Tucson, USA.Figure 3. 15 minutes time window clusters.Time step 2.2

Beneti, C., Calvetti, L., Jusevicius, M. and Gin,R. 2004: The Integration of Radar,Lightning and Satellite Information forThunderstorm Analysis and Nowcasting.ILDC, Helsink, Finland.Bradley, P. S., Fayyad, U. M. and Reina, C. A.1998: Scaling EM (Expectation-Maximization) Clustering to LargeDatabases. Microsoft Technical Report.MSR-TR-98-35.Fayyad, U., Piatetsky-Shapiro, G. and Smyth,P. 1996: The KDD Process for ExtractingUseful Knowledge from Volumes of Data.Knowledge Discovery in CommunicationsOf The ACM.Vol. 39, No. 11.Politi, J., Stephany, S., Domingues, M. O.,Mendes Jr, O. 2004: Uma metodologiapara representação espaço-temporal deocorrências de descarga nuvem-solo.Revista Brasileira de Meteorologia.(submitted)Politi, J.; Stephany, S.; Domingues, M. O.;Mendes Jr, O. 2004: Implementação deum Ambiente para Mineração de DadosAplicado ao Estudo de NúcleosConvectivos. IV WORCAP. São José dosCampos, Brazil.Witten, I. H., Frank, E. 2005: Data mining :practical machine learning tools andtechniques – 2nd ed. Elsevier, 525 pp.Lakshmanan, V., and Stumpf, G. J. 2005: AReal-Time Learning Technique to PredictCloud-to-Ground Lightning. FourthConference on Artificial IntelligenceApplications to Environmental SciencesJ5.6, San Diego, USAMsdn. Acessed in November 2005. URL:http://msdn2.microsoft.com/enus/library/ms174879.aspx3