An efficient mechanism for Matching multiple patterns on XML Streams

More documents

Recommendations

Info

Predicates may also evaluate a boolean expressiongiven by combinations of relational operators (=,!=, , =), the assignment operator (=:), logicaloperators (|, &, !), brackets ((, )), variable references(using $) or the current value from the XMLdata stream (if an operand is left empty). Variable referencesmay be complex as they also need to care <strong>for</strong>template references. For example $foo.bar[3].dingidentifies the value of the field ding of the 3. elementin the sequence bar, where bar is a field of the scalarfoo. The definition of TMPL has a set of rules thatgovern the exact evaluation of predicates. We will notgo into more detail in this article. Please refer to thedocumentation found at the website [6].3.5 Allowing Mixed ContentThe language allows <strong>for</strong> matching of mixed content;values are allowed to appear unrestricted among theXML tags and not only as a single child of an element.Opposite to other processors, the TMPL engine treatsleading and trailing white space as empty content andstrips it from the data.4 Changes in the Evaluation RuntimeImplementation of the stream analysis runtime haschanged considerably, as shown in figure 1. The evaluationengine is interpreting EFSM template matchingautomata using a proprietary binary <strong>for</strong>mat or GXLenabling a more flexible handling of templates, e.g. itis now possible to add or remove templates during runtimeand to combine automata. Per<strong>for</strong>mance of thisapproach using only a single template is inferior tothe <strong>for</strong>mer compiled version, but superior when severaltemplates are matched in parallel.4.1 Automata GenerationThe <strong>for</strong>malism <strong>for</strong> generated automata had to supportcontext (which may be values from the data stream,TMPL declared variables and constants, or internalvariables used by the evaluation engine) and transitionsthat may depend on any of them. These premiseslead to the adoption of an EFSM <strong>for</strong>malism. Morespecifically we choose to extend the one proposed byHenniger and Neumann (see [7]) because of the abovementioned features, the additional support <strong>for</strong> internalɛ transitions and the ability <strong>for</strong> transitions to changethe complete context of an automaton.<strong>An</strong> EFSM is defined as a tuple(S, C, I, O, T, s 0 , c 0 ), where S is a finite set ofstates, C the context 4 , I a non-empty finite setof input events including ɛ, and O a non-emptyfinite set of output events. T denotes the transition4 All possible values of a finite set of variablesrelation from current state, context and input eventto the next state, modified context and output event(T ⊆ S × C × I × O × S × C). The two last elementss 0 and c 0 describe the initial state and context values.Every TMPL component (element, content, attribute,template reference, . . . ) is translated to oneor more states that are combined to a graph using<strong>for</strong>ward and reverse transitions. A <strong>for</strong>ward transitionis understood as to be leading towards the endstate, whereas a reverse transition is used when a pathcannot completed, e.g. when a parent XML elementcloses. As an example consider the following template<strong>for</strong> searching a (hypothetical) music database:template fooMusicList {}const integer c = 10;float ranking;template tSingle[] single;template tAlbum[] album;$c]>[=:ranking]single*album*The template fooMusicList would be translated toan EFSM similar 5 to the one shown in figure 2. Astraight <strong>for</strong>ward path through the generated automatonis marked with bold transition lines. Several detailsmight be associated with a transition: contextconditions which are enclosed in square brackets, actions<strong>for</strong> changing the context in curly brackets, an inputevent from the XML Stream (e.g. START, END,ATTRIBUTE, . . . ) and an output event or none inthe case of an ɛ transition.The first two states are used to initialise constantsand output variables; the matching processstarts in the START and stops in the END statewhere a matching pattern has been identified whichwill be reported back to the analysing application.For every element the generator constructstwo states that match corresponding XML startand end elements (e.g. MATCH AUTHOR andMATCH AUTHOR END); <strong>for</strong> attributes the automatonhas a state that first registers each attribute’sname and then uses a single state to match all of them,this is similar <strong>for</strong> content. Template references aretranslated as single states.5 Some details emitted <strong>for</strong> brevity of examples.
SET_ATTRIBUTESTART[author]{SET(name,Foo Fighters)SET(titles,>c)}MATCH_2_ATTRIBUTESSET_CONTENTLIST_END[false]ATTRIBUTE{CHECK()}LIST_END[true]INIT_CONSTANTINIT_VARIABLESTARTMATCH_AUTHORMATCH_REG_ELEMENTSTART[fan*][=TOP+1]{SET(:=ranking)}START ENDMATCH_CONTENTCONTENT[true]MATCH_REG_ENDEND[=TOP]MATCH_MUSICMATCH_SINGLE_STAR{SET_CONST(c,10)}{SET_VAR(single,LIST)SET_VAR(album,LIST)SET_VAR(ranking,FLOAT)}END[=TOP]END[=TOP]START[music][=TOP+1]MATCHED[tAlbum][=TOP]END[=TOP]MATCHED[tSingle][=TOP]4.2.1 Parallel Execution of TemplatesThe basic idea behind this is to combine several templates,this is a standard procedure in graph theory:by using the cross product a combined automaton maybe calculated (please refer to [8]). It is still inevitableto check conditions <strong>for</strong> all transitions; the per<strong>for</strong>mancegain is solely the difference between the time one needsto execute a single state transition or several ones.The downside to this approach is in the exponentialincrease in states and transitions. For exampletake two EFSM S 1 and S 2 with p i number of states andq i number of transitions: a resulting EFSM <strong>for</strong> S 1 ×S 2would have p 1·p 2 states and at least p 1·q 2 +q 1·p 2 transitions.A combination of ten EFSM with ten stateseach (e.g. templates having three elements and twocontent predicates) would end up with 10 billion (10 10 )states and far more transitions. As the authors of [9]point out, a possible solution to this dilemma is to uselazy construction principles <strong>for</strong> the automata.4.2.2 Lazy Automata ConstructionEven if the hypothetical number of states of a combinedEFSM seems to grow to exorbitant numbers onlya very small portion of these states would ever be used.If an automaton is build in a lazy fashion by constructingnew states at runtime only when they are needed,this number may be decrease dramatically. For examplewe combined twelve templates with altogether 350states using a lazy construction principle and endedup with an EFSM of only 169 states <strong>for</strong> a specific datastream after further optimisations that reduced redundantor empty states and transitions.END[=TOP]MATCH_ALBUM_STAREND[=TOP]MATCH_AUTHOR_ENDMATCHED[tAlbum][=TOP]END[=TOP]ENDTEMPLATE_MATCH()Figure 2. Automaton <strong>for</strong> template fooMusicList4.2 OptimisationWith the interpreting approach in place, several newoptimisations have become possible. One of the mostimportant ones was the combination of several automatain one larger EFSM.4.2.3 State and Transition ReductionThere are several ways to further reduce the numberof states and transitions when combining automata:Merging of ɛ transitions <strong>An</strong>y number of statesthat are connected solely by ɛ transitions aremerged to two states with a single connecting ɛtransition.Removal of initialisation states Initialisationfunctionality does not need to be considered bythe combination algorithm but may be executedseparately.Removal of default self-transitions When no inputevent selects a transition an EFSM needs tostay in its current state. This can be made defaultbehaviour and then does not need to be made explicitby a self-transition.Grouping transitions By grouping transitions witha common condition (e.g. same nesting level or
Page 1 and 2: AN EFFICIENT MECHANISM FOR MATCHING
Page 3: 3.2 Declarative Type SystemThe intr
Page 7: This is a restriction of the TMPL r

An efficient mechanism for Matching multiple patterns on XML Streams

Create successful ePaper yourself

Delete template?

Save as template?