Drill Down Till You Drop: OLAP Without Windows - NESUG

More documents

Recommendations

Info

Sample DataA sample demonstration database is used in the exampleswhich consists of 35,000 observations measuring salesfigures collected from various cities in the USA andCANADA. Each observation contains an individual salesamount generated by a specific SALESPERSON for aparticular DATE, COUNTRY, REGION,STATE/PROVINCE, and CITY.Review of PROC SUMMARYPROC SUMMARY is a SAS procedure which canproduce various summary statistics for all numericvariables in a data set. This procedure is well suited toOLAP and drill down applications since it can produceoutput which leads to many different kinds of analyses,both within the same dimension or across differentdimensions. In PROC SUMMARY terminology thevariables listed on the CLASS statement take on attributesof Dimensions, and variables on the VAR statement aretreated as Facts. Data statistics pertaining to Facts canthen be pre-summarized, and then be available foranalysis with minimum impact upon computation time.PROC SUMMARY produces separate summary statisticsfor various subgroups of variables on the CLASSstatement. A subgroup is defined for each individualCLASS variable, as well as for each combination ofCLASS variables. Thus, 3 variables which are listed onthe CLASS statement will generate 8 separate summarysubgroups.The output from PROC SUMMARY is a special data setcontaining summary statistics which have been calculatedfor each variable listed on the VAR statement. Importantpoints regarding this output data set are:• Each observation contains a special _TYPE_ variablewhich identifies the subgroup to which the summarystatistics apply. By examining the _TYPE_ variableas a bitmap variable you can “decode” a specificsubgroup and determine which variables on theCLASS statement define that subgroup.• Each observation contains a _FREQ_ variablerepresenting the number of observations for thatcurrent subgroup.• The observations also contains the CLASS variableswhich represents the subgroup. If a CLASS variabledoes not contribute to the subgroup it is representedas missing.Let’s use the Sample Data Set as an example and set up aPROC SUMMARY with the following code:2PROC SUMMARY data=drill noprint;CLASS city state region;VAR sales;output out=temp sum=;The output dataset “temp” contains 8 observations whichrepresent sales summaries for all the observations in“drill” broken down by every combination of CITY,STATE, and REGION (Table 1). To see how the_TYPE_ variable works, we will look at the _TYPE_variable as a bit string with each bit representing avariable in the CLASS statement. If that bit is “on”, thevariable it represents is included in the subgroup. Therightmost CLASS variable is represented by therightmost bit, the next adjacent CLASS variable to the leftis represented by the next bit to the left, etc. To arrive at ainteger value for _TYPE_ , you multiple the value of eachbit by 2 raised to the K power, where K represents the bitposition, with 0 being the rightmost position andincreasing by 1 for each bit to the left. Sum all of theresults for every CLASS variable as computed above toarrive at a final value for the _TYPE_ variable.The 8 different _TYPE_ variables corresponding to thefollowing subgroups are defined below:Table 1_TYPE_SUBGROUPBITSTRING OFCITY STATE REGION1 REGION 0 0 12 State 0 1 03 Region by State 0 1 14 City 1 0 05 Region by City 1 0 16 State by City 1 1 07 Region by State by City 1 1 10 TOTAL 0 0 0Example:Using Table 1 Above: The Subgroup “Region” iscomputed as:1*2 0 + 0*2 1 + 0*2 2 = 1while Subgroup “State by City” is computed as:0*2 0 + 1*2 1 + 1*2 2 = 6A _TYPE_ variable equal to 0 always represents statisticsrepresenting the total for the entire dataset. For eachdistinct _TYPE_ there will be as many elements in asubgroup as there are levels or combinations of levels forthe contributing CLASS variables. For example, If oursample data set contained only two states: NJ, and MD,
and contained only three cities, Baltimore, Newark, andAnnapolis, there would be two observations correspondingto _TYPE_=2, three observations corresponding to_TYPE_=4, and three observations corresponding to_TYPE=6.Selection of the Dimensions and DrillHierarchySelection of the proper dimensions for the drill downinvolves careful study of analysts needs and anunderstanding of the relationship among the entitieswhich the data represents. In many cases, the drillhierarchy may follow an organizations standard way ofdoing business, but in other cases, the data can be minedfirst using techniques such as CHAID and CART in orderto find significant dimensions for analysis. The entireprocess of determining the needs of the analyst can not beoveremphasized, since the drill down model shouldprovide a meaningful application for the analyst. If not,the model can end up being too simplistic, complex, orsimply wrong from a business point of view.Our sample data will use a Time/Geography Drillhierarchy. Examining the relationship among the levelsthere is a natural relationship as follows:drill downDATE→ COUNTRY→ REGION→STATE/PROVINCE→ CITYdrill upCITY→ STATE/PROVINCE→ REGION→ COUNTRY→ DATEIn other more complex cases there may not be a naturalorder, or the relationship may be a one-to-manyrelationship. In this case more analysis would be needed.Specification of the PROC SUMMARYNext, we will set up a PROC SUMMARY specifying thehierarchy we have established. However, we must takecare when setting up the CLASS statement, since theorder in which the variables appear is important. In orderto take advantage of numerical properties of the _TYPE_variable, we will specify the drill down order in theCLASS statement from right to left. Therefore thestatement for this example becomes:PROC SUMMARY data=drill noprint;CLASS city state region country date ;VAR sales;output out=temp sum=;Note that we also need to select an analysis variable aswell as a summary statistic. Since drill down applicationsneed to access summary statistics at each level of the drillhierarchy, we need to specify SUM= on the outputstatement. We will also specify SALES as the variablewhich is to be summed.Creating the Optimized Drill DownDatasetAs mentioned earlier, the output of PROC SUMMARYwill generated observations for each combination ofCLASS variables listed on the CLASS statement. Eventhough PROC SUMMARY drastically reduces the numberof observations in the output dataset, as compared to theoriginal dataset, in a drill down hierarchy we are onlyinterested in a subset of all possible subgroups. We willexamine the output dataset “temp” to see whichobservations correspond to the drill hierarchy at eachlevel. Those observations will be kept, while others willbe discarded.• We need the summary for DATE by itself(_TYPE_=1) since this is the starting level for ourdrill down.• The next level of drill down is to the COUNTRYlevel. In this case, the required summary would bethe one corresponding to _TYPE_=3 (DATE byCOUNTRY). We will include DATE as part of thissubgroup since we will already have drilled downfrom the starting DATE level, and need the DATEvariable as a way to provide a link to the previouslevel. If we examine all of the other subgroups whichcontain COUNTRY we will find a summary forCOUNTRY by itself, as well as subgroups containingcombinations of variables such as COUNTRY byCITY, COUNTRY by STATE by CITY, etc. Some ofthese summaries will be discarded, since we are onlyinterested in that summary relevant to positionswithin the drill down hierarchy.• If we continue to work through the drill hierarchy tothe next level, we see that the REGION level requires_TYPE_=7 (DATE by COUNTRY by REGION),since by drilling down to the REGION level, we willhave already drilled down on DATE and thenCOUNTRY. By the same rational given for theprevious level, we can discard the other _TYPE_variables containing REGION not relevant to our drilldown. (For example: REGION by CITY will be3
Page 4 and 5: discarded since REGION is never sel
Page 6: • The month which is being select
Page 9 and 10: down database has been built, it is

Drill Down Till You Drop: OLAP Without Windows - NESUG

Create successful ePaper yourself

Delete template?

Save as template?