Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

speed up deleting million of records from a table with one filter Expand / Collapse
Author
Message
Posted Thursday, November 8, 2012 12:14 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Wednesday, October 16, 2013 12:29 AM
Points: 3, Visits: 54
hi all,
I want to delete million of records from a table with only one filter of varchar(1) column
for that which one is best option.....
Post #1382289
Posted Thursday, November 8, 2012 12:48 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 7:00 AM
Points: 13,615, Visits: 10,499
What do you mean with filter?
Something like this?

DELETE FROM myTable WHERE columnA = 'B'

ps: why define a VARCHAR(1) column? What is wrong with CHAR(1). If you do have millions of rows, you're literally throwing storage away!




How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1382300
Posted Thursday, November 8, 2012 1:29 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Wednesday, October 16, 2013 12:29 AM
Points: 3, Visits: 54
hi
It's actually char(1) column only.I am not asking abt that.
I am asking batch processing is better or any other option will be better.
Because I am going to delete million of records it's time consuming process.so that I am asking...
Post #1382316
Posted Thursday, November 8, 2012 1:39 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 7:00 AM
Points: 13,615, Visits: 10,499
kuppurajdpm (11/8/2012)
hi
It's actually char(1) column only.I am not asking abt that.
I am asking batch processing is better or any other option will be better.
Because I am going to delete million of records it's time consuming process.so that I am asking...


And I asked something about the filter. It's important that we know exactly what you're trying to achieve.




How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1382320
Posted Thursday, November 8, 2012 2:02 AM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Tuesday, April 16, 2013 8:34 AM
Points: 23, Visits: 128
Simplest solution is to use a WITH. e.g.


WITH results ( <select> ) DELETE FROM results –
Post #1382332
Posted Thursday, November 8, 2012 8:07 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 7:20 AM
Points: 13,298, Visits: 12,149
If you want to prevent locking your table for an extended period of time you can do this in batches.

--This will ensure that the value of @@ROWCOUNT for the loop will be > 0
select 1

while @@ROWCOUNT > 0
begin
with DeleteList as (select top 1000 columnA from MyTable order by [Clustered Index])
--order by clustered index so the engine should not have to revisit any given page more than neccesary.
delete DeleteList
end



_______________________________________________________________

Need help? Help us help you.

Read the article at http://www.sqlservercentral.com/articles/Best+Practices/61537/ for best practices on asking questions.

Need to split a string? Try Jeff Moden's splitter.

Cross Tabs and Pivots, Part 1 – Converting Rows to Columns
Cross Tabs and Pivots, Part 2 - Dynamic Cross Tabs
Understanding and Using APPLY (Part 1)
Understanding and Using APPLY (Part 2)
Post #1382514
Posted Thursday, November 8, 2012 5:01 PM
SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Yesterday @ 12:55 PM
Points: 2,033, Visits: 3,041
It depends. Often it's faster just to keep the desired rows rather than delete all the undesired. That is, copy the good rows to another table, trunc the table, copy the good rows back in.

Otherwise you can DELETE in batches as Sean suggests ... but don't forget the filter on the DELETE statement itself .


SQL DBA,SQL Server MVP('07, '08, '09)
"In America, every man is innocent until proven broke!" Brant Parker
Post #1382771
Posted Thursday, November 8, 2012 5:48 PM


SSC Eights!

SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!

Group: General Forum Members
Last Login: Tuesday, August 12, 2014 12:40 PM
Points: 945, Visits: 1,771
ScottPletcher (11/8/2012)
It depends. Often it's faster just to keep the desired rows rather than delete all the undesired. That is, copy the good rows to another table, trunc the table, copy the good rows back in.

Otherwise you can DELETE in batches as Sean suggests ... but don't forget the filter on the DELETE statement itself .


If im only keeping a small number of rows i will insert the rows into a new table but instead of truncating and reinserting the rows i will rename the tables. this will have much less down time as i dont have to reinsert the records into the original table after the truncate. once the tables are renamed i can keep the old table around to make sure every thing is working fine then just drop the table. Personally i think this method has a little more redundancy as the data is never gone until im sure i have every thing i need.



For faster help in answering any problems Please read How to post data/code on a forum to get the best help - Jeff Moden for the best way to ask your question.

For performance Issues see how we like them posted here: How to Post Performance Problems - Gail Shaw

Need to Split some strings? Jeff Moden's DelimitedSplit8K
Jeff Moden's Cross tab and Pivots Part 1
Jeff Moden's Cross tab and Pivots Part 2

Jeremy Oursler
Post #1382784
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse