SQL Server Advanced Query Tuning
SQL Server Advanced Query Tuning
Hands-On-Lab
Page 1 of 31
SQLMaestros Channels
Web : www.SQLMaestros.com
YouTube : https://www.youtube.com/user/SQLMaestros/videos
RSS : http://feeds.feedburner.com/SQLMaestros_AmitBansal
Twitter : www.twitter.com/SQLMaestros
LinkedIn : https://www.linkedin.com/showcase/14630987
Telegram : https://t.me/sqlmaestros
FB : www.facebook.com/SQLMaestros
Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the example companies, organizations,
products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-
mail address, logo, person, place, or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under
copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying,
recording, or otherwise), or for any purpose, without the express written permission of eDominer Systems.
The names of manufacturers, products, or URLs are provided for informational purposes only and eDominer makes no representations and warranties, either expressed, implied, or statutory,
regarding these manufacturers or the use of the products with any Microsoft technologies. The inclusion of a manufacturer or product does not imply endorsement of eDominer of the
manufacturer or product. Links are provided to third party sites. Such sites are not under the control of eDominer and eDominer is not responsible for the contents of any linked site or any
link contained in a linked site, or any changes or updates to such sites. eDominer is not responsible for webcasting or any other form of transmission received from any linked site. eDominer is
providing these links to you only as a convenience, and the inclusion of any link does not imply endorsement of eDominer of the site or the products contained therein.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any
written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. This lab
document (under the HOLs service by SQLMaestros) is just a learning document that enables you to learn and practice a topic/subject step-by-step, to be practiced in your test environment.
Microsoft, Excel, Office, and SQL Server, Azure are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
You hereby confirm and state that you will be the sole recipient of SQLMaestros Hands-On-Labs, depending on whatever you have purchased. You understand that you have purchased the lab
for a lifetime use but the IP remains with eDominer Systems (parent co of SQLMaestros). You confirm that you will not share the lab document(s) or any part of it with any other individual and
/or group and/or company and/or any entity, be it known to me or unknown to me. You specifically confirm that you will not give the downloaded/purchased labs or even a part of it to your
company or your team. Further, you will not reproduce, or make multiple copies or store or introduce into a retrieval system, or transmit in any form or by any means (electronic, mechanical,
photocopying, recording, or otherwise), or for any purpose. You understand that you are allowed to make one and only one backup for safekeeping/DR purposes. You will not share the backup
link with any individual or any entity. Sharing the labs with anyone will be a gross violation of “terms of use”. If you are a trainer/coach/facilitator/teacher/instructor or are involved in teaching
in any capacity, you confirm that you will not use the lab documents or share them with your class students or participants. In summary, you understand that the lab documents are only for
your self-learning, for your personal use. You also understand and agree that any violation of the above, will cause irreparable harm/damage to the SQLMaestros brand, the Author & eDominer
Systems as a company and all its brands. Under such circumstances, eDominer Systems shall be entitled to such injunctive or equitable relief as may be deemed proper by a court of competent
jurisdiction against you.
Table of Contents
Terms of Use, Copyright & Intellectual Property ............................................................................................................................................................................. 2
Table of Contents ............................................................................................................................................................................................................................. 3
Before You Begin.............................................................................................................................................................................................................................. 4
Estimated time to complete this lab ............................................................................................................................................................................................ 4
Objectives: ................................................................................................................................................................................................................................... 4
Lab Setup Requirements .............................................................................................................................................................................................................. 4
Prerequisites ................................................................................................................................................................................................................................ 4
Lab Scenario ................................................................................................................................................................................................................................. 4
Tips to complete this lab successfully .......................................................................................................................................................................................... 5
Exercise 1: Use Cases Of TOP ........................................................................................................................................................................................................... 6
Scenario........................................................................................................................................................................................................................................ 6
Summary .................................................................................................................................................................................................................................... 16
Exercise 2: Query Tuning Techniques ............................................................................................................................................................................................ 17
Scenario...................................................................................................................................................................................................................................... 17
Summary .................................................................................................................................................................................................................................... 30
Other learning opportunities from us that might interest you. .................................................................................................................................................... 31
Objectives:
After completing this lab, we will be able to learn:
• You must have SQL Server 2012 Standard/Developer/Enterprise/Evaluation edition or higher. Click here to download SQL Server evaluation edition
• You must have SQLMaestros sample databases. Click here to download SQLMaestros sample databases
Prerequisites
Before executing this lab:
Lab Scenario
This lab talks about a very common yet difficult problem, finding TOP n rows per group. The problem may seem easy, however in this lab we’ll discuss multiple
ways to solve this problem and compare the different ways on performance to find out the fastest way to solve the problem. In the first exercise, we’ll talk
about TOP use cases and in the second exercise we’ll talk about the Top n rows per group problem and it’s different solutions.
• All lab files are located in SQL Server Advanced Query Tuning folder
• The script(s) are divided into various sections marked with ‘Begin’, ‘End’ and ‘Steps’. As per the instructions, execute the statements between
particular sections only or for a particular step
• Read the instructions carefully and do not deviate from the flow of the lab
• Practice this lab only in your test machine/environment. Do not run this lab in your production environment
Scenario
In this exercise, we will learn few use cases of TOP keyword available in SQL Server.
Launch SQL Server 1. Click Start | All Programs | SQL Server 2012 | SQL Server Management Studio
Management Studio 2. In the Connect to Server dialog box, click Connect
Setup Review and execute the statement(s) in following setup section. The setup section will create
• Source table
• Target table
• Inserts few records into Source table using hol.fn_numbers function
---------------
--Begin: Setup
---------------
IF OBJECT_ID('Source','U') IS NOT NULL
DROP TABLE Source
GO
CREATE TABLE Source
(
col1 INT,
col2 INT,
col3 INT
);
GO
IF OBJECT_ID('Target','U') IS NOT NULL
DROP TABLE Target
GO
CREATE TABLE Target
(
col1 INT ,
col2 INT ,
col3 INT
);
GO
-- Insert data into source table
INSERT INTO Source
SELECT n ,
n + 1,
n * 2
FROM hol.fn_numbers(100);
---------------
--End: Setup
---------------
Execute the statement(s) Execute the following statement(s) to find 3 latest orders placed by the subscribers.
SubTotal,
ServiceTax,
OrderDate,
ModifiedDate,
DiscountPercentage,
TotalCost
FROM hol.TransactionDetails
Explanation: The above query retrieved top three records from the hol.TransactionDetails table, in order to get latest three
orders placed by the subscriber, but SQL server cannot guarantee that those three are the latest orders, as we did not specify
the ORDER BY clause in the above query.
Execute the statement(s) Execute the following statement(s) to find the 3 latest order places by the subscribers.
Explanation: The above query retrieved top three records from the hol.TransactionDetails table, in order to get latest three
orders placed by the subscriber, but SQL server cannot guarantee that those three are the latest orders, as we specified the
non-unique column in the ORDER BY.
Execute the statement(s) Execute the following statement(s) to find 3 latest orders placed by the subscriber.
Explanation: The above query retrieves top three records from hol.TransactionDetails table, in order to get latest three orders
placed by the subscriber. OrderDate column in ORDER BY clause is to find latest orders based on order date. The above query
also used TransactionID in ORDER BY clause, which makes this query deterministic as the TransactionID column contains
unique values.
Execute the following Execute the following statement(s) to find most recent one percent orders placed by a subscriber.
statement(s)
--Step 4: Execute the following statement(s) to find Most-Recent One Percent of Orders
SELECT TOP (3) PERCENT
TransactionID,
SubscriptionID,
PaymentMethod,
SubTotal,
ServiceTax,
OrderDate,
ModifiedDate,
DiscountPercentage,
TotalCost
FROM hol.TransactionDetails
ORDER BY OrderDate DESC ,
TransactionID DESC;
GO
Note: The image is a trimmed snapshot of original result set. The original result set contains more number of rows.
Explanation: The above query retrieved the top three percent of most recent orders placed by the subscribers.
Find n most recent orders Execute the following statement(s) to find n most recent orders.
SELECT TOP ( @n )
TransactionID,
SubscriptionID,
PaymentMethod,
SubTotal,
ServiceTax,
OrderDate,
ModifiedDate,
DiscountPercentage,
TotalCost
FROM hol.TransactionDetails
ORDER BY OrderDate DESC ,
TransactionID DESC;
Explanation: By using above query, we can find n number of recent orders placed by the subscribers, simply by providing a
valid numeric value for @n. In the above query, the value of n is 2, so the query retrieved TOP 2 latest orders placed by the
subscribers.
Find Average number of Execute the following statement(s) to get the average number of monthly orders.
month orders
--Step 6: Execute the following statement(s) to find the average number of monthly orders
SELECT COUNT(*) / ( DATEDIFF(MONTH, MIN(OrderDate), MAX(OrderDate)) + 1 ) AS AvgNoOfMonthlyOrders
FROM hol.TransactionDetails
Explanation: The above query is used to calculate the number of orders placed by the subscriber on an average. In this case,
COUNT(*) will return 100000 , as hol.TransactionDetails contains 100000 records. DATEDIFF function will find the number of
months. We will use this query in the next step. The average number of monthly orders is the count of orders divided by one
more than the difference in months between the maximum and minimum order dates. Because 100000 orders in the table
were placed during a period of 94 months.
Find most recent average Execute the following statement(s) to find the monthly average number of orders.
number of monthly orders
--Step 6: Execute the following statement(s) to find most recent average number of monthly orders
SELECT TOP ( ( SELECT COUNT(*) / ( DATEDIFF(MONTH, MIN(OrderDate),
MAX(OrderDate)) + 1 )
FROM hol.TransactionDetails ) )
TransactionID,
SubscriptionID,
PaymentMethod,
SubTotal,
ServiceTax,
OrderDate,
ModifiedDate,
DiscountPercentage,
TotalCost
FROM hol.TransactionDetails
ORDER BY OrderDate DESC, TransactionID DESC;
Explanation: The number of rows returned are the monthly average number of orders. The number of rows are 1052.
Note: The above image is a trimmed snapshot of original result set. The original result set contains more number of rows.
Inserting records Execute the following statement(s) to insert 10 records into target table from Source table.
--Step 7: Execute the following statement(s) to insert 10 records into target table
INSERT TOP ( 10 )
INTO [Target]
SELECT col1 ,
col2 ,
col3
FROM [Source];
Explanation: The above query inserts 10 records into Target table from Source table. There is no guarantee, which records will
get inserted as there is no ORDER BY clause.
View records Execute the following statement(s) to view the records in target table.
--Step 8: Execute the following statement(s) to view the records in target table
SELECT *
FROM dbo.Target;
Execute the statement(s) Execute the following statement(s) to insert 10 records into Target table.
Explanation: The above query fails to insert the records into Target table and throws an error, as we cannot use ORDER BY
clause with INSERT, DELETE and UPDATE statements with TOP.
UPDATE with TOP Execute the following statement(s) to update the top 10 records in Source table.
--Step 10: Execute the following statement(s) to perform update operation using TOP
WITH CTE_UPD
AS ( SELECT TOP(10) *
FROM [Source]
)
UPDATE CTE_UPD
SET col2 = col2 + 1;
Explanation: The above query updates the TOP 10 rows in the Source table using CTE. CTE comes into rescue, whenever we
want to perform UPDATE or DELETE operations in the presence of TOP filter.
DELETE with TOP Execute the following statement(s) to delete the top 10 records from the Source table.
--Step 11: Execute the following statement(s) to perform delete operation using TOP
;WITH CTE_DEL
AS ( SELECT TOP(10) *
FROM [Source]
)
DELETE FROM CTE_DEL;
GO
Explanation: The above query deletes the TOP 10 rows in the Source table using CTE.
Close all the query Close all the query windows ( ) and if SSMS asks to save changes, click NO
windows
Summary
In this exercise, we have learned
Problem Statement: “Find the most recent Hands-On-Labs downloaded by each subscriber using different kind of approaches and compare the performance
of each approach”.
Scenario
In this exercise, we will learn to solve the given problem statement using various methods such as JOINs, Subqueries, APPLY operators and will compare the
performance of each querying technique.
Setup Review and execute the following statement(s) in setup section. The setup section will
• UPDATE the DownloadDate of few subscribers to prepare the data for the exercise
• Creates a relevant index
---------------
--Begin: Setup
---------------
USE SQLMaestros
GO
/* Prepare Data */
UPDATE hol.DownloadStatistics
SET DownloadDate = GETDATE()
Execute the statement(s) Execute the following statement(s) to find the downloads by the subscriber with ID 16.
--Step 1: Execute the following statement(s) to find the download details of subscriber with ID 16
SELECT *
FROM hol.DownloadStatistics
WHERE SubscriberID = 16
ORDER BY DownloadDate DESC ,
DownloadID DESC;
Explanation: The above query retrieves the download details of SubscriberID 16. Observe that, the subscriber has downloaded
4 labs. Our motive is to find recent most lab downloaded by the subscriber. We will use this query in the next step to find the
recent most download details of the subscriber with ID 16.
Note: The above image is a trimmed snapshot of original result set. The original result set contains more number of rows.
Execute the statement(s) Execute the following statement(s) to find the most recent download by a subscriber with ID 16.
--Step 2: Execute the statement(s) to find the most recent download by a subscriber with ID 16
SELECT d1.SubscriberID ,
d1.DownloadID ,
d1.HolID ,
d1.DownloadDate
FROM hol.DownloadStatistics AS d1
WHERE d1.DownloadID = ( SELECT TOP ( 1 )
d2.DownloadID
FROM hol.DownloadStatistics AS d2
WHERE d1.SubscriberID = d2.SubscriberID
ORDER BY d2.DownloadDate DESC ,
d2.DownloadID DESC
) AND D1.SubscriberID=16
ORDER BY d1.SubscriberID
Explanation: The subquery used in the above query arranges the records based on DownloadDate in descending order.
DownloadID column in the ORDER BY list acts as a tie breaker in case multiple downloads happened at the same time. Then
selecting the TOP 1 record from the subquery gives us the most recent download by the subscriber with ID 16.
Find single most recent Execute the following statement(s) to find the single most recent download by all the subscribers.
download by the subscriber
--Step 3: Execute the following statement(s) to find the most recent download by all the subscribers
SELECT d1.SubscriberID ,
d1.DownloadID ,
d1.HolID ,
d1.DownloadDate
FROM hol.DownloadStatistics AS d1
Explanation: The subquery used in the above query arranges the records based on DownloadDate in descending order.
DownloadID column in the ORDER BY list acts as a tie breaker in case multiple downloads happened at the same time. Observe
the condition at WHERE clause, the query filter on SubscriberID of DownloadStatistics table. Then selecting the TOP 1 record
from the subquery gives us the most recent download by all the subscribers.
Note: The above image is a trimmed snapshot of original result set. The original result set contains more number of records.
--Step 4: Execute the following statement(s) to find n most recent downloads by all the subscriber
--Turn on Actual Execution Plan
DECLARE @numdownloads INT;
SET @numdownloads = 1;
SELECT d1.SubscriberID ,
d1.DownloadID ,
d1.HolID ,
d1.DownloadDate
FROM hol.DownloadStatistics AS d1
WHERE d1.DownloadID IN ( SELECT TOP ( @numdownloads )
d2.DownloadID
FROM hol.DownloadStatistics AS d2
WHERE d1.SubscriberID = d2.SubscriberID
ORDER BY d2.DownloadDate DESC ,
d2.DownloadID DESC )
ORDER BY d1.SubscriberID;
Note: The above image is a trimmed snapshot of original result set. The original result set contains 300 rows.
In the execution plan, hover the mouse pointer over Index Seek and observe the number of executions
Observation: The above query retrieved the single most recent downloads by each subscriber as the value of n is 1. The
execution plan says that the query has been executed 100002 times to retrieve only 10000 records, because
Hol.DownloadStatistics table contains 100002 rows and the subquery executes once per each row. If we make the query to
execute once per each subscriber, then we can reduce the number of executions to 10000, which will be our next step.
Execute the statement(s) Execute the following statement(s) to find the single most recent download of each subscriber.
Note: The above image is a trimmed snapshot of original result set. The original result set contains more number of rows.
Observation: The above query retrieved the single most recent download details for each subscriber using a co-related
subquery. Observe the condition at WHERE clause, the query filter the records based on SubscriberID from hol.Subscribers
table and SubscriberID from hol.DownloadStatistics table. To get the remaining download information such as DownloadID,
HolID and DownloadDate, JOIN the above query with hol.DownloadStatistics table, which will be our next step.
Execute the following statement(s) to find the single most recent download by all the subscribers.
--Step 6: Execute the following statement(s) to find single most download by each subscriber
SELECT O.DownloadID ,
O.SubscriberID ,
O.HolID ,
O.DownloadID
FROM ( SELECT S.SubscriberID ,
( SELECT TOP ( 1 )
O2.DownloadID
FROM hol.DownloadStatistics AS O2
WHERE O2.SubscriberID = S.SubscriberID
ORDER BY O2.DownloadDate DESC ,
O2.DownloadID DESC
) AS LatestDownload
FROM hol.Subscribers AS S
) AS LatestDownload
JOIN hol.DownloadStatistics AS O ON O.DownloadID = LatestDownload.LatestDownload;
Note: The above image is a trimmed snapshot of original result set. The original result set contains more number of rows.
In the execution plan, hover the mouse over Index Seek (NonClustered) to observe the number of Executions
Observation: The above query retrieved the single most recent download per subscriber. The execution plan says that, the
query has been executed only 10000 times to retrieve 1000 records because. This query is executed less number of times than
the query in Step 4 to retrieve the same number of records. This is because, this time we are comparing Subscriber ID’s from
hol.DownloadStatistics table and hol.Subscribers table. The problem with this query is, it cannot be extended to retrieve n
number of most recent downloads per subscriber. If we replace the integer 1 in the above query with the value greater than
1, then the query will throw an error as subquery is not permitted to return more than one value when it is used as an
expression.
Execute the statement(s) Execute the following statement(s) to find the single most recent downloads per subscriber.
--Step 7: Execute the following statement(s) to find the single most download per subscriber
SELECT S.SubscriberID ,
A.DownloadID ,
A.HolID ,
A.DownloadDate
FROM hol.Subscribers AS S
CROSS APPLY ( SELECT TOP ( 1 )
D.DownloadID ,
D.HolID ,
D.DownloadDate
FROM hol.DownloadStatistics AS D
WHERE D.SubscriberID = S.SubscriberID
ORDER BY D.DownloadDate DESC ,
D.DownloadID DESC
) AS A
GO
Note: The above image is a trimmed snapshot of original result set. The original result set contains more number of rows.
In the execution plan, hover the mouse pointer over Index Seek (NonClustered) to observe the number of executions
Observation: The above query retrieved the single most recent downloads per subscriber. The execution plan says that, the
query has been executed 1000 times to retrieve 1000 records. Unlike the query at Step 6, this query can be extended to find
n number of recent downloads as we used CROSS APPLY operator. The CROSS APPLY operator will give the records from the
left expression, if the right-side expression produces a result set. For example, The SubscriberID will be included in the result
set only if that subscriber downloaded a lab. In the next step, we will learn one more way to achieve the same.
Create necessary indexes Execute the following statement(s) to create necessary indexes.
Execute the statement(s) Execute the following statement(s) to find the three most recent downloads per subscriber.
--Step 9: Execute the following statement(s) to find the three most recent downloads per subscriber
;WITH CTE
AS ( SELECT ROW_NUMBER() OVER ( PARTITION BY SubscriberID ORDER BY DownloadDate DESC,
DownloadID DESC ) AS rn ,
SubscriberID,
HolID,
DownloadDate,
DownloadID
FROM hol.DownloadStatistics
)
SELECT CTE.SubscriberID,
CTE.DownloadID,
CTE.HolID,
CTE.DownloadDate
FROM CTE
WHERE rn <= 3;
GO
Note: The above image is a trimmed snapshot of original result set. The original result set contains more number of rows.
Observation: The above query retrieved the three most recent downloads per subscriber using ROW_NUMBER window
function inside CTE. The ROW_NUMBER function assigns a row number to each record partition by SubscriberID and order by
DownloadDate in descending order. We selected the records which have the row number greater than or equals to 3, so that
it retrieves three most recent downloads per subscriber.
CTE.DownloadID ,
CTE.HolID ,
CTE.DownloadDate
FROM CTE
WHERE rn <= 3;
GO
--Query 2: Using CROSS APPLY
SELECT S.SubscriberID ,
A.DownloadID ,
A.HolID ,
A.DownloadDate
FROM hol.Subscribers AS S
CROSS APPLY ( SELECT TOP ( 3 )
D.DownloadID ,
D.HolID ,
D.DownloadDate
FROM hol.DownloadStatistics AS D
WHERE D.SubscriberID = S.SubscriberID
ORDER BY D.DownloadDate DESC ,
D.DownloadID DESC
) AS A;
GO
Observation: The query with ROW_NUMBER function takes 16% of batch cost, whereas the query using CROSS APPLY takes
84% of batch cost. In this case, the query using ROW_NUMBER function performs better as the hol.DownloadStatistics table
contains 1 lakh records. The query with CROSS APPLY function will perform better, if table has less number of rows.
Close all the query Close all the query windows ( ) and if SSMS asks to save changes, click NO
windows
Summary
In this exercise, we have learned
Data Platform Summit is Asia’s largest Data, Analytics & AI learning www.DPS10.com
event. Whole week of deep-dive training takes place in Bangalore each www.DataPlatformSummit.in
year in the month of August/September. World’s best speakers and
delegates join from more than 20 countries.
At SQLShighra.com you get quick SQL Server Tips and know-how www.SQLShighra.com
related to SQL Server Internals, Troubleshooting and Performance
Tuning. These are short & casual videos, mostly Amit Bansal’s SQL
brain dump and he records them anywhere, anytime (not literally).
A team of specialists on SQL Server & Microsoft Data Platform, Follow/Join SQL Server Discussions:
SQLMaestros offers affordable learning solutions and also offers SQL RSS Subscription, Telegram, YouTube, Twitter,
Server health check & baselining service. LinkedIn, Facebook
Video Courses. Hands-On-Labs. Learning Kits.
*The above information is constantly updated here: www.eDominer.com/learning