Eric lippert - Amazon Web Services

09
Housekeeping databases
If you have transactional tables where records get inserted very
frequently and unless you remove older records from the table, you
run out of space within weeks; then it is a challenge to periodically
purge such tables, without bringing down the tables. If you have
a 24x7 website, then it becomes very difficult to find a slot where
you can take an outage and do a purge that takes on an average of
15-30 mins to finish. During purge operations, the table gets locked
and all other queries on that table times out. So, you need to find
a way to purge records consuming the least amount of database
resource.
delete from @BatchIds
Set @rowCount = @@rowCount;
Print ‘Deleted ‘ + convert(varchar(10), @rowcount) + ‘
rows...’
waitfor delay ‘00:00:05’
End;
go
Checkpoint 30; -- simple recovery, if full - put tlog
backup here
Go
There are various ways to do this but we found the following
approach having the least footprint on the database:
Declare @batchsize int; Set @batchsize = 1000;
set transaction isolation level read uncommitted; set
nocount on;
declare @temp_rows_to_delete table (ID int primary key)
insert into @temp_rows_to_delete select ID FROM
VeryLargeTable WHERE ID < 4000
Declare @rowCount int; Set @rowCount = 1;
declare @BatchIds table (id int primary key)
While @rowCount > 0
Begin
-- Pick the IDs to delete and remove them from #temp_rows
delete top(@batchsize) from @temp_rows_to_delete
OUTPUT deleted.ID into @BatchIds
delete from VeryLargeTable WHERE ID IN (SELECT ID FROM @
BatchIds)
The idea here is to do a steady small batch of delete at a time until
all the records we want to purge are deleted. This query can go
on for hours, not a problem. But during that time, there will be no
significant stress on the database to cause other queries to timeout
or degrade significantly.
First we set the isolation level. This is absolutely a key thing to
do. This isolation level tells SQL Server that the following query
does not need locking. It can read dirty data, does not matter. So,
there’s no reason for SQL Server to lock records in order to ensure
the query gets properly committed data. Unless you are a bank, you
should put this in all your stored procs and you will see 10 times
better throughput sometimes from your database, for read queries.
Next we read the row IDs from the source table and store in a table
variable. This is the only time we are going to do a SELECT on the
source table so that SQL Server does not have to scan the large
table again and again.
Then we keep picking 1000 row IDs at a time and delete the rows
from the table. After each delete, we give SQL Server 5 seconds to
rest and flush the logs.
10
Tuning database indexes
Over the time, data in tables get fragmented as data gets inserted,
updated and deleted. If you leave the tables alone, the indexes will
get slower and slower resulting in slower queries. If you have
millions of records on a table and you start having significant
fragmentation on the tables, over the time, queries will get so slow
that they will start timing out.
In order to keep the tables fast and lean, you need to regularly
rebuild the indexes. Here’s a SQL snippet that will run through
all the indexes on the database and see if the indexes are heavily
fragmented. If they are, then it will issue an index rebuild on them.
Ref: http://www.mssqltips.com/sqlservertip/1165/managing-sqlserver-database-fragmentation/
10 | DNCmagazine www.dotnetcurry.com