IJIES-2008 VOLUME 1 ISSUE 4 - Index of

International Journal ofIntelligent Engineering & Systemshttp://www.inass.org/Signal-based Approach to Anomaly Detection in IDS SystemsŁukasz Saganowski 1 , Michał Choraś 2,∗ , Rafał Renk 3 , Witold Hołubowicz 4 ,1 Institute of Telecommunications University of Technology & Life Sciences, Bydgoszcz,S. Kaliskiego 7, 85-791 Bydgoszcz,, Poland2 ITTI Ltd., Rubiez 11, Poznań, Poland and Institute of Telecommunications University of Technology & LifeSciences, Bydgoszcz,3 ITTI Ltd. and Adam Mickiewicz University, Poznań, Poland4 ITTI Ltd. and Adam Mickiewicz University, Poznań, PolandAbstract: In this paper we present our original methodology, in which Matching Pursuit is used for networks anomalyand intrusion detection. The architecture of anomaly-based IDS based on signal processing is presented. We proposeto use mean projection of the reconstructed network signal to determine if the examined trace is normal or attacked.Experimental results confirm the efficiency of our method in worm detection scenario. The practical usability of theproposed approach in the intrusion detection tolerance system (IDTS) in the INTERSECTION project is presented.Keywords: Intrusion Detection System (IDS), Anomaly detection, Heterogeneous networks security1. IntroductionIntrusion Detection Systems (IDS) are based onmathematical models, algorithms and architectural solutionsproposed for correctly detecting inappropriate,incorrect or anomalous activity within a networkedsystems [1].Intrusion Detection Systems can be classified as belongingto two main groups depending on the detectiontechnique employed:(i) anomaly detection(ii) signature-based detection.Anomaly detection techniques rely on the existenceof a reliable characterization of what is normaland what is not, in a particular networking scenario.More precisely, anomaly detection techniques basetheir evaluations on a model of what is normal, and∗ Corresponding author.Email addresses: luksag@utp.edu.pl,michal.choras@itti.com.pl, rafal.renk@itti.com.pl,holubowicz@amu.pl.classify as anomalous all the events that fall outsidesuch a model [2].In this paper our original methodology for networksanomaly and intrusion detection based on MatchingPursuit is presented. In Section 2 general overviewof the proposed architecture and decision block detailsare shown. In section 3 the motivation for signalprocessing methodologies used in intrusion detectionis given. In section 4 Matching Pursuit algorithmand base function of the proposed dictionary design isshown. Experimental results and conclusion are giventhereafter.The major contribution of this paper, is the intrusion/anomalydetection algorithm based on the MatchingPursuit. As to our best knowledge, we have notmet any other IDS system based on matching pursuit.Even though our Matching-Pursuit anomaly detectionapplication is not working in a real time now, itis used in the off-network layer of INTERSECTIONIntrusion Detection Tolerance System (IDTS) as describedin Section 6.International Journal of Intelligent Engineering and Systems 4 (2008) 18–24 18

Figure 1. IDS system block diagramFigure 2. IDS decision block diagram2. General overview of our anomaly-based IDSapproachBy profiling the properties of normal network trafficand modeling intrusions or unwanted traffic asanomalies, it is possible to detect the occurrence ofsuch events within reasonable time so to activate reactionand response procedures. Determining the normalbehavior model, however, is a difficult task due tothe presence of different trends in data, which mightbe influenced by the time of day, the day of week andseasonal variations.In our approach we store ”normal” traces in a referencedatabase. Normal traces represent traffic fromdays, we are sure no attacks occurred. These referencetraces are compared to current, examined traces.Current traces may be either sniffed from traffic or forexperimental purposes may represent old attacks (sothat the ground truth is known).The general overview of our intrusion detectionsystem is presented in Figure 1. The overview of adecision block is explained in Figure 2.Signal processing techniques have found applicationin Network Intrusion Detection Systems becauseof their ability to detect novel intrusions and attacks,which cannot be achieved by signature-basedapproaches. It has been shown that network trafficpresents several relevant statistical properties whenanalyzed at different levels (e.g. self-similarity, longrange dependence, entropy variations, etc.) [3]. Approachesbased on signal processing and on statisticalanalysis can be powerful in decomposing the signalsrelated to network traffic, giving the ability to distinguishbetween trends, noise, and actual anomalousevents. Wavelet-based approaches, maximum entropyestimation, principal component analysis techniques,and spectral analysis, are examples in this regardwhich have been investigated in the recent years bythe research community [4]-[8].A powerful analysis, synthesis, and detection tool inthis field is represented by the wavelets. Indeed, timeandscale-localization abilities of the wavelet transform,make it ideally suited to detect irregular trafficpatterns in traffic traces. Recently many waveletbasedmethods for detection of attacks have beentested and documented. Some are based on the continuouswavelet transform analysis, most of them howeverrefer to the discrete wavelet transformation andthe multiresolution analysis [3].However, Discrete Wavelet Transform provides alarge amount of coefficients which not necessarily reflectrequired features of the network signals.Therefore, in this paper we propose anothersignal processing and decomposition method foranomaly/intrusion detection in networked systems.We developed original Anomaly Detection Type IDSalgorithm based on Matching Pursuit.3. Motivation for signal-based IDSSignal processing techniques have found applicationin Network Intrusion Detection Systems becauseof their ability to detect novel intrusions and attacks,which cannot be achieved by signature-basedapproaches. It has been shown that network trafficpresents several relevant statistical properties whenanalyzed at different levels (e.g. self-similarity, longInternational Journal of Intelligent Engineering and Systems 4 (2008) 18–24 19

ange dependence, entropy variations, etc.) [3]. Approachesbased on signal processing and on statisticalanalysis can be powerful in decomposing the signalsrelated to network traffic, giving the ability to distinguishbetween trends, noise, and actual anomalousevents. Wavelet-based approaches, maximum entropyestimation, principal component analysis techniques,and spectral analysis, are examples in this regardwhich have been investigated in the recent years bythe research community [4]-[8].A powerful analysis, synthesis, and detection tool inthis field is represented by the wavelets. Indeed, timeandscale-localization abilities of the wavelet transform,make it ideally suited to detect irregular trafficpatterns in traffic traces. Recently many waveletbasedmethods for detection of attacks have beentested and documented. Some are based on the continuouswavelet transform analysis, most of them howeverrefer to the discrete wavelet transformation andthe multiresolution analysis [3].However, Discrete Wavelet Transform provides alarge amount of coefficients which not necessarily reflectrequired features of the network signals.Therefore, in this paper we propose anothersignal processing and decomposition method foranomaly/intrusion detection in networked systems.We developed original Anomaly Detection Type IDSalgorithm based on Matching Pursuit.4. Intrusion Detection System based on MatchingPursuit4.1. Introduction to Matching PursuitMatching Pursuit signal decomposition was proposedby Mallat and Zhang [9].Matching Pursuit is a greedy algorithm that decomposesany signal into a linear expansion of waveformswhich are taken from an overcomplete dictionary D.The dictionary D is an overcomplete set of base functionscalled also atoms.D = {α γ : γ ∈ Γ} (1)where every atom α γ from dictionary has normequal to 1:‖α γ ‖ = 1 (2)Γ represents set of indexes for atom transformationparameters such as translation, rotation and scaling.Signal s has various representations for dictionaryD. Signal can be approximated by set of atoms α kfrom dictionary and projection coefficients c k :s =|D|−1∑n=0c k α k (3)To achieve best sparse decomposition of signal s(min) we have to find vector c k with minimal normbut sufficient for proper signal reconstruction. MatchingPursuit is a greedy algorithm that iteratively approximatessignal to achieve good sparse signal decomposition.Matching Pursuit finds set of atoms α γksuch that projection of coefficients is maximal. At firststep, residual R is equal to the entire signal R 0 = s.R 0 = 〈α γ0 ,R 0 〉α γ0 + R 1 (4)If we want to minimize energy of residual R 1 wehave to maximize the projection |〈α γ0 ,R 0 〉|. At nextstep we must apply the same procedure to R 1 .R 1 = 〈α γ1 ,R 1 〉α γ1 + R 2 (5)Residual of signal at step n can be written as follows:R n s = R n−1 s − 〈 R n−1 s|α γk〉αγk (6)s =Signal s is decomposed by set of atoms:N−1∑n=0〈α γk |R n s〉α γk + R n s (7)Algorithm stops when residual R n s of signal islower then acceptable limit.4.2. Our Approach to Intrusion DetectionAlgorithmIn basic Matching Pursuit algorithm atoms are selectedin every step from entire dictionary which hasflat structure. In this case algorithm causes significantprocessor burden. In our coder dictionary with internalstructure was used.Dictionary is built from:— Atoms,— Centered atoms,Centered atoms groups such atoms from D that areas more correlated as possible to each other. To calculatemeasure of correlation between atoms functiono(a,b) can be used [2] .o(a,b) =√1 −( |〈a,b〉|‖a‖ 2‖b‖ 2) 2(8)The quality of centered atom can be estimated accordingto (9):International Journal of Intelligent Engineering and Systems 4 (2008) 18–24 20

0.30.20.10-0.1-0.2-0.3-0.4-0.5450 500 550 6000.40.30.20.10-0.1-0.2-0.3-0.4460 480 500 520 540 560Figure 3. Example dictionary atomsO k,l = 1 ∑o ( )A|LP k,l |c(i) ,W c(k,l)i∈LP k,l(9)LP k,l is a list of atoms grouped by centered atom. O k,lis mean of local distances from centered atom W c(k,l)to the atoms A c(i) which are strongly correlated withA c(i) .Centroid W c(k,l) represents atoms A c(i) which belongsto the set i ∈ LP k,l . List of atoms LP k,l shouldbe selected according to the Equation 10:max o ( )A c(i) ,W c(k,l) ≤i∈LP k,lmin o ( )A c(t) ,W c(k,l) (10)t∈D\LP k,lIn the proposed IDS solution 1D real Gabor basefunction (Equation 11) was used to build dictionary[10]-[12].α u,s,ξ,φ (t) = c u,s,ξ,φ α( t − u )cos(2πξ(t − u) + φ) (11)swhere:α(t) = 1 √ se −πt2 (12)c u,s,ξ,φ - is a normalizing constant used to achieveatom unit energy,In order to create overcomplete set of 1D base functionsdictionary D was built by varying subsequentatom parameters: Frequency ξ and phase φ, Positionu, Scale s [13].Base functions dictionary D was created with using10 different scales (dyadic scales) and 50 differentfrequencies.In Figure 3 example atoms from dictionary D arepresented.5. Experiments and ResultsIn our experiments we decided to detect worm attacks.We tested our algorithms on normal and attackedtraces to evaluate if our method is capable ofdetecting known worms.Similarly to the work by Dainotti et al. [14] wetested the efficiency of our algorithms on Slammer andWitty worms. Slammer worm spread in 2003, whileWitty spread in March 2004.In our experiments we use TCP and UDP packetsof Slammer and Witty made available by the WIDE-MAWI and CAIDA projects [15][16].In this paper we will show our algorithm tested onattacked and normal traces.The attacked traces represent traffic (TCP andUDP packets) from March 20th (Witty) (Figure 4)and March 25th (Slammer) (Figure 5).The normal traces represent traffic from March 6thand March 13th (Figures 6-7).The calculated values of Matching Pursuit MeanProjection for our test traces (normal and attacked)are presented in Tables 1-2.In tables 1 Matching Pursuit Mean Projection valuesfor TCP packets are presented. In tables 2 MatchingPursuit Mean Projection values for UDP packetsare given, respectively.Table 1. Mean Projection values calculated for test TCPtracesTCP TraceMP25.03.2004 (Slammer) 62020.03.2004 (Witty) 6676.03.2004 45313.03.2004 373Decision block of our system is based on the MatchingPursuit Mean Projection values. As presented inFigure 1 we calculate difference Diff between examinedand normal traces stored in a reference database.If the value Diff is larger than a certain threshold tour application signalizes the attack/anomaly.In the experiments shown here, in the case of Wormattacks, our application was set to t = 30%, whichmeans that if Matching Pursuit Mean Projection differsmore than 30% from the reference normal tracesthe attack should be detected.As presented in Tables 1-2 mean projection valuesdiffer significantly and our IDS application successfullydetects Witty and Slammer worms. In our experimentswe can report 100% worm detection for TCPand UDP packets with no false alarms. However, so farwe tested our method on a limited number of traces.We decided to use known and benchmark traces andworms first. Now we extensively test our method witha larger number of real-networks anonimized tracesas well as with the generated traffic traces.International Journal of Intelligent Engineering and Systems 4 (2008) 18–24 21

0 100 200 300 400 500 600 700 800 90080001400700012006000100050008004000600300040020001000200000 100 200 300 400 500 600 700 800 900Figure 4. Traces attacked by Witty worm from March 20th - TCP (left) and UDP (right).8000100078009007600740080072007007000680060066005006400400620060000 100 200 300 400 500 600 700 800 9003000 100 200 300 400 500 600 700 800 900Figure 5. Traces attacked by Slammer worm from March 25th - TCP (left) and UDP (right).4400400420035040003003800360025034002003200150300028000 100 200 300 400 500 600 700 800 9001000 100 200 300 400 500 600 700 800 900Figure 6. Normal trace from March 6th - TCP (left) and UDP (right).500050048004504600440040042003504000380030036002503400200320030000 100 200 300 400 500 600 700 800 9001500 100 200 300 400 500 600 700 800 900Figure 7. Normal trace from March 13th - TCP (left) and UDP (right).International Journal of Intelligent Engineering and Systems 4 (2008) 18–24 22

Table 2. Mean Projection values calculated for test UDPtracesUDP TraceMP25.03.2004 (Slammer) 8220.03.2004 (Witty) 1276.03.2004 3213.03.2004 40In the article our developments in feature extractionfor Intrusion Detection systems are presented. Weshowed that Matching Pursuit may be considered asvery promising methodology which can be used innetworks security framework. Upon previous experimentswe concluded that Matching Pursuit Mean Projectiondiffers significantly for normal and attackedtraces [17]. Hereby we verified that Matching PursuitMean Projection ADS approach successfully detectsSlammer and Witty worms.The major contributions of this paper is a novel algorithmfor detecting anomalies based on signal decomposition.In the classification/decision module weproposed to use developed matching pursuit featuressuch as mean projection. We tested and evaluated thepresented features and showed that experimental resultsproved the effectiveness of our method.The proposed Matching Pursuit signal based algorithmapplied for anomaly detection IDS will be usedas detection/decision module in the INTERSECTIONProject security-resiliency framework for heterogeneousnetworks.AcknowledgementThe research leading to these results has receivedfunding from the European Community’s SeventhFramework Programme (FP7/2007-2013) under grantagreement no. 216585 (INTERSECTION Project).ReferencesFigure 8. IDS decision block diagram6. Practical Usability of the Proposed MethodSignal-based anomaly detection type IDS will beused as the secondary detection/decision module tosupport real-time IDS. Such approach is proposedfor off-network layer of the INTERSECTION framework.The operator will have a chance to observe the resultsof signal-based IDS in a near real-time in orderto trigger or stop the reaction of real-time IDS.Such approach will both increase the security (less detectedanomalies/attacks) and increase the tolerance(less false positives). The overview of the MatchingPursuit IDS role in the INTERSECTION architectureis given in Figure 8.7. Conclusion[1] Esposito M., Mazzariello C., Oliviero F., RomanoS.P., Sansone C., Real Time Detection of NovelAttacks by Means of Data Mining Techniques. ICEIS(3) 2005: 120-127.[2] Esposito M., Mazzariello C., Oliviero F., RomanoS.P., Sansone C., Evaluating Pattern RecognitionTechniques in Intrusion Detection Systems. PRIS2005: 144-153.[3] FP7INTERSECTION INfrastructure for heTErogeneous,Reislient, Secure, Complex, Tightly Inter-OperatingNetworks Project Description of Work.[4] C.-M. Cheng, H.T.Kung, K.-S. Tan, Use of spectralanalysis in defense against DoS attacks, IEEEGLOBECOM 2002, pp. 2143-2148.[5] P. Barford, J. Kline, D. Plonka, A. Ron, Asignal analysis of network traffic anomalies,ACMSIGCOMM InternetMeasurement Workshop 2002.[6] P. Huang, A. Feldmann, W. Willinger,A nonintrusive,wavelet-based approach to detectingnetwork performance problems, ACM SIGCOMMInternet Measurement Workshop, Nov. 2001.[7] L. Li, G. Lee, DDos attack detection and wavelets,IEEE ICCCN03, Oct. 2003, pp. 421-427.[8] A. Dainotti, A. Pescape, G. Ventre, Wavelet-basedDetection of DoS Attacks, 2006 IEEE GLOBECOM- Nov 2006, San Francisco (CA, USA).[9] S. Mallat and Zhang Matching Pursuit with timefrequencydictionaries. IEEE Transactions on SignalProcessing., vol. 41, no 12, pp. 3397-3415, Dec 1993.International Journal of Intelligent Engineering and Systems 4 (2008) 18–24 23

[10] J.A. Troop. Greed is Good: Algorithmic Resultsfor Sparse Approximation. IEEE Transactions onInformation Theory., vol. 50, no. 10, october 2004 r.[11] R. Gribonval Fast Matching Pursuit with a MultiscaleDictionary of Gaussian Chirps. IEEE Transactionson Signal Processing., vol. 49, no. 5, may 2001.[12] P. Jost, P. Vandergheynst and P. Frossard Tree-BasedPursuit: Algorithm and Properties. Swiss FederalInstitute of Technology Lausanne (EPFL),SignalProcessing Institute Technical Report.,TR-ITS-2005.013, May 17th, 2005.[13] Andrysiak T., Choraś M., Image Retrieval Based onHierarchical Gabor Filters, InternationalJournal Applied Mathematics and Computer Science(AMCS), vol. 15, no. 4, 471-480, 2005.[14] A. Dainotti, A. Pescape, G. Ventre, Worm TrafficAnalysis and Characterization, Proceedings of ICC,IEEE CS Press, 1435-1442, 2007.[15] WIDE Project: MAWI Working Group TrafficArchive at tracer.csl.sony.co.jp/mawi/[16] The CAIDA Dataset on the Witty Worm - March19-24, 2004, Colleen Shanon and David Moore,www.caida.org/passive/witty.[17] Renk R., Saganowski Ł., Houbowicz W., ChoraśM., Intrusion Detection System Based on MatchingPursuit, in Proc. Intelligent Networks and IntelligentSystems, ICINIS ’08, 213-216, IEEE CS Press, 2008.International Journal of Intelligent Engineering and Systems 4 (2008) 18–24 24