Advanced SQL - JMU - James Madison University

Recursion in SQL; Window FunctionsFCDB 10.2Dr. Chris MayfieldDepartment of Computer ScienceJames Madison UniversityApr 15, 2014

Special topics poll30 responses as of 1:00 PM todayAssertions; Triggers (more about transactions) 19%Full Text Search (queries over text documents) 17%Advanced SQL (recursion, analytical queries) 26%MySQL; SQLite (a quick tour of other DBMS’s) 26%Other (exploits, functions, php, project work day) 11%Apr 15, 2014 Recursion in SQL; Window Functions 2 of 18

More about subqueriesSee FCDB 6.3.2–6.3.4

SQL operators for relationsLet R be a relation and t be a tuple from R◮ EXISTS R◮ t IN R◮ t > ALL R◮ t > ANY R“Find the departments of the courses Suri has taken.”SELECT deptFROM takeWHERE stud_pid IN (SELECT pidFROM studentWHERE name = ’Suri ’);Use NOT to negate EXISTS, IN, ALL, ANYApr 15, 2014 Recursion in SQL; Window Functions 4 of 18

Correlated subqueries“Find course names that have been used for two or more courses.”SELECT nameFROM courses AS firstWHERE name IN (SELECT nameFROM coursesWHERE dept first . deptOR number first . number );Read this query from the inside out!Apr 15, 2014 Recursion in SQL; Window Functions 5 of 18

WITH clause (FCDB 10.2)Basic syntax:WITH R AS Example relation:◮ Flights(airline, frm, to, departs, arrives)Example query:WITH den_flights AS (SELECT * FROM FlightsWHERE frm = ’DEN ’)SELECT * FROM den_flightsORDER BY departs ;Apr 15, 2014 Recursion in SQL; Window Functions 6 of 18

Common table expressionsDefine temporary tables that exist for one query◮ WITH can involve SELECT, INSERT, UPDATE, or DELETE◮ Can be attached to SELECT, INSERT, UPDATE, or DELETEFor example:WITH moved_rows AS (DELETE FROM productsWHERE " date " >= ’2010 -10 -01 ’AND " date " < ’2010 -11 -01 ’RETURNING *)INSERT INTO products_logSELECT * FROM moved_rows ;http://www.postgresql.org/docs/9.1/static/queries-with.htmlApr 15, 2014 Recursion in SQL; Window Functions 7 of 18

Real world exampleDBLP = Database of CS journals and proceedings◮ 2.2 million publications◮ 1.1 million authorsHome page: http://dblp.uni-trier.de/See also: http://dl.acm.org/dl.cfmApr 15, 2014 Recursion in SQL; Window Functions 8 of 18

Erdős numbersWITH e1 AS (-- Erdos number is 1SELECT DISTINCT b. fullnameFROM author AS a-- same paper , but different authorJOIN author AS b ON a. dblp_key = b. dblp_keyAND a. fullname b. fullnameWHERE a. fullname = ’Paul Erdos ’)-- Erdos number is 2SELECT DISTINCT d. fullnameFROM e1-- first get all papers of e1 authorsJOIN author AS c ON e1. fullname = c. fullnameJOIN author AS d ON c. dblp_key = d. dblp_keyAND c. fullname d. fullname-- excluding 0 and 1WHERE d. fullname ’Paul Erdos ’AND d. fullname NOT IN ( SELECT fullname FROM e1 );Apr 15, 2014 Recursion in SQL; Window Functions 9 of 18

Recursive queries using CTE’sNot to be confused withCareer & Technical Education

Recursive relations in SQLRECURSIVE modifer allows WITH queries to refer to their own output-- Result is 5050WITH RECURSIVE t(n) AS (VALUES (1)UNION ALLSELECT n+1 FROM t WHERE n < 100)SELECT sum (n) FROM t;General form:1. Non-recursive term2. UNION or UNION ALL3. Recursive termApr 15, 2014 Recursion in SQL; Window Functions 11 of 18

Recursive query evaluation1. Evaluate the non-recursive term◮ Include all rows in the query result◮ If UNION, eliminate duplicate rows◮ Also place them in a working table2. While the working table is not empty◮ Evaluate the recursive term using working table◮ If UNION, eliminate duplicates of any previous row◮ Add rows to result and create intermediate table◮ Replace working table with the intermediate tableStrictly speaking, this process is iteration not recursion!Apr 15, 2014 Recursion in SQL; Window Functions 12 of 18

Flight example from book-- transitive closure of flightsWITH RECURSIVE Reaches (frm , to) ASSELECT frm , toFROM FlightsUNIONSELECT R1.frm , R2.toFROM Reaches AS R1 , Reaches AS R2WHERE R1.to = R2.frm)-- all cities reachable from DenverSELECT to FROM ReachesWHERE frm = ’DEN ’;Note:◮ You may skip section 10.2.2 on mutual recursion◮ It’s interesting, but beyond the scope of CS 474Apr 15, 2014 Recursion in SQL; Window Functions 13 of 18

More Advanced SQLSaved the best for last

Analytical queriesCalculate a running total◮ Show the cumulative salary within a department row by rowFind percentages within a group◮ Show the percentage of the total salary paid to an individualCompute a moving average◮ Average the current row’s value with the previous N rowsPerform ranking queries◮ Show the relative rank of each salary within a departmentTop-N queries◮ Find the top n sales by regionSource: http://wiki.postgresql.org/wiki/SQL2008 windowing queriesApr 15, 2014 Recursion in SQL; Window Functions 15 of 18

Window functionsPerform a calculation across related rows◮ Partition: which rows are related◮ Order: how to sort each partitionExample:-- sort by salary in each deptSELECT depname , empno , salary ,rank () OVER ( PARTITION BY depname ORDER BY salary DESC )FROM empsalary ;Window functions only allowed in SELECT and ORDER BY clauses◮ Defined over output of FROM, WHERE, GROUP BY, and HAVINGApr 15, 2014 Recursion in SQL; Window Functions 16 of 18

Example OVER clauses-- average salary in each deptSELECT depname , empno , salary ,avg ( salary ) OVER ( PARTITION BY depname )FROM empsalary ;-- running total of salariesSELECT depname , empno , salary ,sum ( salary ) OVER ( ORDER BY salary )FROM empsalary ;-- GROUP BY without groupingSELECT depname , empno , salary ,sum ( salary ) OVER ()FROM empsalary ;http://www.postgresql.org/docs/9.1/static/tutorial-window.htmlApr 15, 2014 Recursion in SQL; Window Functions 17 of 18

Other window functionsSELECT depname , empno , salary ,sum ( salary ) OVER w, -- and other aggregate functionsrow_number () OVER w, -- from 1 to number of rows in wrank () OVER w, -- rows with same value get same rankFROM empsalaryWINDOW w AS ( PARTITION BY depname ORDER BY salary DESC );◮ There are many more options for OVER clauses◮http://www.postgresql.org/docs/9.1/static/sqlexpressions.html#SYNTAX-WINDOW-FUNCTIONS◮ List of general-purpose window functions◮http://www.postgresql.org/docs/9.1/static/functions-window.htmlApr 15, 2014 Recursion in SQL; Window Functions 18 of 18