An efficient mechanism for Matching multiple patterns on XML Streams

This paper discusses XML stream data processingfrom the perspective of TMPL, a language thatwas specifically designed <strong>for</strong> stream analysis and notadopted from a document based approach. TMPL wasintroduced in [1], further in<strong>for</strong>mation on related workin this field may be found in this paper, as well.3 Language ExtensionsTMPL is a context-sensitive language describing <strong>patterns</strong>that are to be matched against a data stream.Syntactical structures of TMPL are specified in asyntax quite similar to XML itself and may consistof elements, attributes and data specifications in the<strong>for</strong>m of so-called predicates. Predicates are denotedusing square brackets and may restrict content (e.g.a statement like a=[>10] would match only if thecorresponding attribute posses a value greater thanten) or assign a value to a variable (e.g. [=:b]would assign the content of the element to avariable of name b). There are additional symbols <strong>for</strong>marking a single unknown element or an element at anunknown depth 2 in the XML data stream. A TMPLfile defines a single module that contains severaltemplates and it is possible to define constants at thebeginning of a template or module scope. Variablesor constants may also be referenced in predicates (e.g.[$c] matches if an unknown element is beencountered that contained the value stored in thevariable referenced by the name of c). Based on theexperience we gained while applying the languageduring the last year, we extended the functionalitywith features we were missing, which either improveexpressiveness by enabling the matching of previouslynot identifiable <strong>patterns</strong> (e.g. sequence matching) orincrease usability of the language by reducing possibletypos or failures on the user’s side (e.g. declarativetype system).To accommodate certain new runtime features itbecame necessary to change the TMPL toolchain. Figure1 depicts the current toolchain in white colour; thegrey part shows the old approach that was directlygenerating Java code from the TMPL abstract syntaxtree (AST) as constructed by the parser. Thishas been dropped in favour of a new intermediary <strong>for</strong>matthat codifies the automata <strong>for</strong> the runtime engineusing an extended finite state machine (EFSM) <strong>for</strong>malismwhich will be described in section 4.1.The basic idea was to be able to use this <strong>for</strong>matin several ways. <strong>An</strong> automaton may be savedin, or loaded from a file using the standard Graph eXchangeLanguage (GXL) <strong>for</strong>mat (<strong>for</strong> more in<strong>for</strong>mationplease refer to the website [5]). Templates in the intermediary<strong>for</strong>mat are used directly with the TMPL2 Stream depth is defined as the nesting level of elementsgxltmplParserAutomataGenerationInterpreterJavaGenerationjavaCompiledRuntimeFigure 1. TMPL tool chainInterpreter, that is the matching runtime engine, tosearch <strong>for</strong> <strong>patterns</strong> in the data stream. Some other optionsnot shown here would be to generate specialisedmatching code directly from the intermediary <strong>for</strong>mat<strong>for</strong> a certain programming language or to use standardvisualisation software to generate pictures of the statemachines.3.1 Syntactical ChangesThe syntax of TMPL has been slightly changedto be better understandable. Explicit comparisonand assignment operators have been introduced (e.g.[=$foo] and [=:bar]) which may be omitted whentheir meaning within a predicate is non-ambiguous.There is a shorthand syntax <strong>for</strong> other syntactical constructsas well, please refer to the website [6] <strong>for</strong> documentationabout these.A major change is the introduction of the rangeelement that offers a clearer definition than the<strong>for</strong>mer placeholder expression. One can use it to matcha sequence of nested elements. As an example considerthe following TMPL code:In this case the range element stands <strong>for</strong> a sequencethat has to begin with a child element of and that must end with an element containing attributeatt and child element b. The range element ismatched successfully when the starting element of thesequence closes.