Tuning SQL for Optimal Performance - Sybase

Agenda• What affects applicationperformance• SQL Optimization and theApplication Lifecycle• How to achieve and maintainmaximum performance• Interview with MichaelBrundige, Sr. Sybase DBAof Brinks

Gartner Study• According to industry experts,60% to 90% of databaseapplication performanceproblems are caused by SQLSQLRelated60%Other40%“Poorly written OLTP application SQL continues to be thelargest contributor to poor database performance.”– Gartner Group

A Corporate Performance Strategy• Hardware Upgrades 0 — 25%• Operating System Settings 0 — 50%• Database Server Parameter Settings 0 — 100%• Database Design 0 — 1000%• Indexes on Database Tables 0 — 1000%• SQL Statements 0 — 100,000%Based on LECCOTECH customer feedback

Sample of Customer ResultsFremont Compensation – resource consumptionOriginal statement: 2.5 hours SQL Expert optimized result: • 1.5 minutesOriginal statement: 4 hours SQL Expert optimized result: • 1.3 minutesOriginal statement: 8 hours SQL Expert optimized result: • 2 hoursMetasolv, Plano, TX – productivityOriginal statement: 15 minutesSQL Expert optimized result: • 3 secondsKentucky Community Technical Colleges – skill setOriginal statement: 1.2 hours SQL Expert optimized result: • 15 seconds(complex statement with 8 table joins)

Sybase SQL Expert: A Performance Assurance StrategyThe Application Life-CycleEnhancements orNew RequirementsDevelopmentBestPerformingCodeMaintainOptimalPerformanceProductionQA & StressMaximumPerformanceBefore Rollout

3-Golden Rules for Writing SQL• Make it work• Make it work right• Make it work fast

The Complex Nature of SQL• Difficult to write SQL for performance• Many ways to write a SQL• Small differences in coding SQL can have great performanceimplications• Need constant tuning due to data and system environmentchanges• Most important factor to system scalability

How to Tune SQL• Understand how the database processes SQL statements• Know database structure• Know data in the database• Different SQL syntax can change performance

How Does ASE Process SQL?Server Receives SQLDataSQLParse, Normalize &Pre-ProcessASE OptimizerDetermines the Query PlanBindExecute

How the ASE Optimizer Works*SQLInternallyRewrites &GeneratesMultipleQuery plansPlan 1Plan 2Plan 3Cost EstimationPlan 1 cost=1000Plan 2 cost=3000Plan 3 cost=500

How the ASE Optimizer Works*SQLInternallyRewrites &GeneratesMultipleQuery plansPlan 1Plan 2Plan 3Can it try every possibleway to rewrite your SQL?Cost EstimationPlan 1 cost=1000Plan 2 cost=3000Plan 3 cost=500

How the ASE Optimizer Works*SQLInternallyRewrites &GeneratesMultipleQuery plansPlan 1 cost=1000Plan 1Plan 2Plan 3Can it try every possibleway to rewrite your SQL?Cost EstimationHow Accurate is theCost EstimationPlan 2 cost=3000Plan 3 cost=500Execution

The Perfect Optimizer?“For any real-life optimizer, no matter how sophisticated, there willalways be a query, a database state and a system state where theoptimizer makes the wrong decision.”Mihnea Andrei (Sybase France) &Patrick Valduriez (Laboratoire d’Informatique de Paris)User-Optimizer Communication, Abstract Plans Tech paper

2 Different SQL’s – Same Query PlanElapsed Time: 0.080 sElapsed Time: 0.080 s

2 Different SQL’s – 2 Different PlansElapsed Time: 0.080 sElapsed Time: 0.110 s

Find the Best Way to Process Your SQL

Find the Best Way to Process Your SQL

Manual Approach – SQL Tuning• Obtain show_plan and analyze output. Look for problemoperations such as full table scansiSQLSET SHOWPLAN ONGOSELECT *FROM EMPLOYEEWHERE EM P_ID > 73712GOSET SHOWPLAN OFFGOQUERY PLAN FOR STATEMENT 1 (at line 1).STEP 1The type of query is SELECT.FROM TABLEEMPLOYEENested iteration.Table Scan.Forward scan.Positioning at start of table.Using I/O Size 2 Kbytes for data pages.With LRU Buffer Replacement Strategy for data pages.Total estimated I/O cost for statement 1 (at line 1): 20600.

Manual Approach – SQL Tuning• Try different ways to write the SQL statement• Techniques to write SQL:– IN, EXISTS, or JOIN– Full table scan, or index scan– JOIN access path– NOT IN, or MINUS– Outer Join, or UNION ALL– IN or OR– Forces?– Temp tables?– ….

Article – Fifty Ways to Quote Your QueryBY C. J. Date, July 1998Database Programming & Design1. SELECT DISTINCT S.SNAMEFROM S, SPWHERE S.S# = SP.S#AND SP.P# = 'P2'“if there are so many ways toformulate such a simple query,how many ways are there goingto be for a complicated one?”2. SELECT DISTINCT S.SNAMEFROM SWHERE S.S# IN( SELECT SP.S#FROM SPWHERE SP.P# = 'P2' )3. SELECT DISTINCT S.SNAMEFROM SWHERE S.S# =ANY( SELECT SP.S#FROM SPWHERE SP.P# = 'P2' )4. SELECT DISTINCT S.SNAMEFROM SWHERE S.S# MATCH( SELECT SP.S#FROM SPWHERE SP.P# = 'P2' )5. SELECT DISTINCT S.SNAMEFROM SWHERE S.S# MATCHUNIQUE( SELECT SP.S#FROM SPWHERE SP.P# = 'P2' )6. SELECT DISTINCT S.SNAMEFROM SWHERE S.S# MATCHPARTIAL( SELECT SP.S#FROM SPWHERE SP.P# = 'P2' )

Manual Approach – SQL Tuning• Time the alternatives in ISQLiSQLSET STATISTICS IO ONGOSET STATISTICS TIME ONGOSELECT *FROM EMPLOYEEWHERE EMP_ID > 73712GOSET STATISTICS IO OFFGOSET STATISTICS TIME OFFGOServer Message: Number 3631, Severity 10Line 1:Total actual I/O cost for this command: 0.Total writes for this command: 0Server Message: Number 3631, Severity 10Line 1:Total actual I/O cost for this command: 0.Total writes for this command: 0Execution Time 0.SQL Server cpu time: 0 ms. SQL Server elapsed time: 3614 ms.Parse and Compile Time 0.SQL Server cpu time: 0 ms.Table: EMPLOYEE scan count 1, logical reads: (regular=1030 apf=0 total=1030),physical reads: (regular=0 apf=0 total=0), apf IOs used=0Server Message: Number 3631, Severity 10Line 1:Total actual I/O cost for this command: 2060.Total writes for this command: 0Execution Time 16.SQL Server cpu time: 1600 ms. SQL Server elapsed time: 7730 ms.(18371 rows affected)Parse and Compile Time 0.SQL Server cpu time: 0 ms.Execution Time 0.SQL Server cpu time: 0 ms. SQL Server elapsed time: 0 ms.Parse and Compile Time 0.SQL Server cpu time: 0 ms.

SQL Expert• Streamlines the SQL optimization process• Proactively identifies problematic SQL without running applications• AI-based SQL transformation generates every possible alternativeand unique query plan• Benchmarks SQL to identify the most efficient alternative for a dbenvironment• Provide the “OPTIMUM” SQL alternative

SQL Optimization MethodologyAnalyzeCapture/ExtractOptimizeMaximizePerformance

SQL Optimization MethodologyAnalyze• Dynamic SQL:Captures SQL throughthe Sybase MonitorServer— SQL MonitorCapture/ExtractMaximizePerformanceOptimize• Embedded SQL:Extracts SQL from ASEdatabase objects (sp’s,views, etc), files,source code(PowerBuilder, etc)— SQL Scanner

SQL Monitor – Capture currently running SQL

SQL Scanner – Extract embedded SQL from db objects & source code

SQL Optimization MethodologyAnalyzeCapture/ExtractOptimize• Analyze in batch the query plan for multiple SQL• Identify potentially problematic operations: Fulltables scans-large tables, many table scans,worktables,…—SQL ScannerMaximizePerformance

SQL Scanner – Analyze and identify problematic query plans

Problematic SQL identified, without running it!

SQL Optimization Methodology• AI based recursive SQLtransformation• Provides unique queryplans• Guarantees semanticallyequivalence• Capture/ExtractRewrites SELECT,SELECT…INTO,INSERT, DELETE andUPDATE• Benchmarks test SQL—Syntactical SQL OptimizerAnalyzeMaximizePerformanceOptimize

What is Recursive SQL TransformationUsing 2 transformation rulesSELECT * FROM AWHERE EXISTS (SELECT ‘x’ FROM BWHERE EXISTS (SELECT ‘x’ FROM CWHERE B.C2=C.C2)WHERE A.C1=B.C1)IN toEXISTSSELECT * FROM AWHERE A.C1 IN (SELECT B.C1 FROM BWHERE EXISTS (SELECT ‘x’ FROM CWHERE B.C2=C.C2 ))EXISTSto INSELECT * FROM AWHERE A.C1 IN (SELECT B.C1 FROM BWHERE B.C2 IN(SELECT C.C2 FROM C))EXISTSto ININ toEXISTSSELECT * FROM AWHERE A.C1 IN (SELECT B.C1 FROM BWHERE EXISTS (SELECT ‘x’ FROM CWHERE B.C2=C.C2 ))SELECT * FROM AWHERE EXISTS (SELECT ‘x’ FROM BWHERE B.C2 IN (SELECT C.C2 FROM C)AND A.C1=B.C1)SELECT * FROM AWHERE A.C1 IN (SELECT B.C1 FROM BWHERE B.C2 IN (SELECT C.C2 FROM C))SELECT * FROM AWHERE EXISTS (SELECT ‘x’ FROM BWHERE B.C2 IN (SELECT C.C2 FROM C)AND A.C1=B.C1)SELECT * FROM AWHERE A.C1 IN (SELECT B.C1 FROM BWHERE EXISTS (SELECT ‘x’ FROM CWHERE B.C2=C.C2))SELECT * FROM AWHERE EXISTS (SELECT ‘x’ FROM BWHERE EXISTS (SELECT ‘x’ FROM CWHERE B.C2=C.C2)AND A.C1=B.C1)

SQL Optimization – In 2 minutes, SQL Expert generated 279 alternatives

Benchmark testing the SQL alternatives against the database –Locate the most–efficient SQL for the ASE environment

Original SQLElapsed Time: 0.763 sFastest SQL AlternativeElapsed time: 0.060 s

Polling QuestionYes or NoHave you used ASE’sAbstract Plans?

Abstract PlanASESQLOptimizerBased on thedatabase statisticsto generate a queryplanExecutionBased on the savedabstract plan togenerate a queryplan

Sybase SQL Expert Facts• Sybase SQL Expert (SSE) is sold as an option to ASE• Windows-based product, Client-based product, no server-sideinstallation• Databases supported:– From version 11.0 to ASE 12.5.0.3– To capture SQL from the Sybase Monitor Server, ASE 11.5 or above required– To use Abstract Plans, ASE 12.0 or above required• Open Client/CT-Lib version 11 or above• Sybase Monitor Server is only needed to capture currently runningSQL; you can use SQL Expert without the Sybase Monitor Server• Non-intrusive– Scanning and optimization process occurs in the PC; no over-head to thedatabase

Key SQL Expert Benefits• Automates current manual tuning efforts• Solves the problem consuming 60–90% of IT resources• Delivers immediate and quantifiable applicationperformance gains• Heightens staff expertise• Increases staff productivity• Establishes benchmarks and performance standardsUse Sybase SQL Expert to ensure onlythe best performing statements are used.

Interview with MichaelBrundige, Sr. SybaseDBA of Brinks