SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Delete 5 lac records from table


Delete 5 lac records from table

Author
Message
VSSGeorge
VSSGeorge
SSCertifiable
SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)

Group: General Forum Members
Points: 7178 Visits: 1592
I have a table where there is a column called nationality having possible values "US" and "Non -US'.
I need to delete the "Non -US' records from my table which amounts to more than 5 lac records, in the best possible way considering the database performance.
Kindly suggest the ways to do that.
John Mitchell-245523
John Mitchell-245523
SSC Guru
SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)

Group: General Forum Members
Points: 131919 Visits: 19201
-- Subsititute x for the number of rows you're happy to
-- delete at a time in order to limit transaction log growth
DELETE TOP(x)
FROM MyTable
WHERE MyColumn = 'Non -US'
-- Rinse and repeat


John
Conficker
Conficker
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1245 Visits: 620
Agree with the DELETE statement, would suggest to use BEGIN TRAN to be on a safe side and then COMMIT
John Mitchell-245523
John Mitchell-245523
SSC Guru
SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)SSC Guru (131K reputation)

Group: General Forum Members
Points: 131919 Visits: 19201
Conficker - Friday, February 9, 2018 4:14 AM
Agree with the DELETE statement, would suggest to use BEGIN TRAN to be on a safe side and then COMMIT

Doesn't make much difference for a single statement. If you're doing this in a lot of batches then you don't want to be doing an explicit COMMIT after each one.

John

andrew gothard
andrew gothard
SSChampion
SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)

Group: General Forum Members
Points: 10562 Visits: 6639
Hi there,
in terms of performance, one thing you'll need - in order to prevent issues with other processes, even when they will not be touching the records in question is lock escalation.
Very briefly:
Initially, your SQL Server engine will look at either taking out row or page locks on the table concerned.
However, locks have a cost in terms of management and resource allocation - they're, to a degree, expensive.
Therefore, after a certain threshold, where the Engine considers locking the individual records / pages in question too expensive, it will escalate the lock to a table lock. This prevents other processes accessing those records, due to an inability to take out their own locks.
The approximate threshold per operation is c. 5000 locks (3000 pre 2005).

So, something based around this;

DECLARE @Rows INT = 1
WHILE @Rows > 0
BEGIN
DELETE TOP <number>
FROM <table>
WHERE <your filter>
SELECT @Rows = @@ROWCOUNT
END
PRINT 'Done'

I'm a DBA.
I'm not paid to solve problems. I'm paid to prevent them.
GilaMonster
GilaMonster
SSC Guru
SSC Guru (902K reputation)SSC Guru (902K reputation)SSC Guru (902K reputation)SSC Guru (902K reputation)SSC Guru (902K reputation)SSC Guru (902K reputation)SSC Guru (902K reputation)SSC Guru (902K reputation)

Group: General Forum Members
Points: 902379 Visits: 48719
If there aren't too many foreign keys and the amount you're deleting is more than the amount of rows you want to keep, it may well be faster to insert the rows you want to keep into a new table, drop the old table and then rename the new one (and put all the keys and indexes back afterwards)

Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability

We walk in the dark places no others will enter
We stand on the bridge and no one may pass


VSSGeorge
VSSGeorge
SSCertifiable
SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)SSCertifiable (7.2K reputation)

Group: General Forum Members
Points: 7178 Visits: 1592
GilaMonster - Friday, February 9, 2018 5:29 AM
If there aren't too many foreign keys and the amount you're deleting is more than the amount of rows you want to keep, it may well be faster to insert the rows you want to keep into a new table, drop the old table and then rename the new one (and put all the keys and indexes back afterwards)

That's a good idea. But should I include it in a transaction? If you can post a sample code, it will be ideal. Thanks

Jeff Moden
Jeff Moden
SSC Guru
SSC Guru (870K reputation)SSC Guru (870K reputation)SSC Guru (870K reputation)SSC Guru (870K reputation)SSC Guru (870K reputation)SSC Guru (870K reputation)SSC Guru (870K reputation)SSC Guru (870K reputation)

Group: General Forum Members
Points: 870295 Visits: 47422
VSSGeorge - Saturday, February 24, 2018 10:57 PM
GilaMonster - Friday, February 9, 2018 5:29 AM
If there aren't too many foreign keys and the amount you're deleting is more than the amount of rows you want to keep, it may well be faster to insert the rows you want to keep into a new table, drop the old table and then rename the new one (and put all the keys and indexes back afterwards)

That's a good idea. But should I include it in a transaction? If you can post a sample code, it will be ideal. Thanks


Can you describe the table a bit more? For example, what are the total number of rows that it has? How many foreign keys does it have pointing at other tables and how many foreign keys point at the table being deleted from? How much total disk space does the table occupy?

All these questions will help tell me if you can do the ol' trick of creating a temporary file group to isolate the good rows on, truncate the original table, and copy the rows back in a minimally logged fashion (for performance and to keep from blowing the log file out), and then dropping the temporary file group to keep from unnecessarily increasing the size of the MDF file. It's nearly the same as what Gail mentioned but has the advantage of not expanding the MDF file any further.

--Jeff Moden

RBAR is pronounced ree-bar and is a Modenism for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column.
If you think its expensive to hire a professional to do the job, wait until you hire an amateur. -- Red Adair

Helpful Links:
How to post code problems
How to post performance problems
Forum FAQs
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum








































































































































































SQLServerCentral


Search