Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


speed up deleting million of records from a table with one filter


speed up deleting million of records from a table with one filter

Author
Message
kuppurajdpm
kuppurajdpm
Forum Newbie
Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)

Group: General Forum Members
Points: 3 Visits: 54
hi all,
I want to delete million of records from a table with only one filter of varchar(1) column
for that which one is best option.....
Koen Verbeeck
Koen Verbeeck
SSCoach
SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)

Group: General Forum Members
Points: 16455 Visits: 13207
What do you mean with filter?
Something like this?

DELETE FROM myTable WHERE columnA = 'B'



ps: why define a VARCHAR(1) column? What is wrong with CHAR(1). If you do have millions of rows, you're literally throwing storage away!



How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at SQLKover.

MCSA SQL Server 2012 - MCSE Business Intelligence
kuppurajdpm
kuppurajdpm
Forum Newbie
Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)

Group: General Forum Members
Points: 3 Visits: 54
hi
It's actually char(1) column only.I am not asking abt that.
I am asking batch processing is better or any other option will be better.
Because I am going to delete million of records it's time consuming process.so that I am asking...
Koen Verbeeck
Koen Verbeeck
SSCoach
SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)

Group: General Forum Members
Points: 16455 Visits: 13207
kuppurajdpm (11/8/2012)
hi
It's actually char(1) column only.I am not asking abt that.
I am asking batch processing is better or any other option will be better.
Because I am going to delete million of records it's time consuming process.so that I am asking...


And I asked something about the filter. It's important that we know exactly what you're trying to achieve.



How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at SQLKover.

MCSA SQL Server 2012 - MCSE Business Intelligence
peterdru401
peterdru401
Grasshopper
Grasshopper (23 reputation)Grasshopper (23 reputation)Grasshopper (23 reputation)Grasshopper (23 reputation)Grasshopper (23 reputation)Grasshopper (23 reputation)Grasshopper (23 reputation)Grasshopper (23 reputation)

Group: General Forum Members
Points: 23 Visits: 128
Simplest solution is to use a WITH. e.g.


WITH results ( <select> ) DELETE FROM results –
Sean Lange
Sean Lange
SSCoach
SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)

Group: General Forum Members
Points: 16582 Visits: 17024
If you want to prevent locking your table for an extended period of time you can do this in batches.


--This will ensure that the value of @@ROWCOUNT for the loop will be > 0
select 1

while @@ROWCOUNT > 0
begin
with DeleteList as (select top 1000 columnA from MyTable order by [Clustered Index])
--order by clustered index so the engine should not have to revisit any given page more than neccesary.
delete DeleteList
end



_______________________________________________________________

Need help? Help us help you.

Read the article at http://www.sqlservercentral.com/articles/Best+Practices/61537/ for best practices on asking questions.

Need to split a string? Try Jeff Moden's splitter.

Cross Tabs and Pivots, Part 1 – Converting Rows to Columns
Cross Tabs and Pivots, Part 2 - Dynamic Cross Tabs
Understanding and Using APPLY (Part 1)
Understanding and Using APPLY (Part 2)
ScottPletcher
ScottPletcher
Hall of Fame
Hall of Fame (3.9K reputation)Hall of Fame (3.9K reputation)Hall of Fame (3.9K reputation)Hall of Fame (3.9K reputation)Hall of Fame (3.9K reputation)Hall of Fame (3.9K reputation)Hall of Fame (3.9K reputation)Hall of Fame (3.9K reputation)

Group: General Forum Members
Points: 3946 Visits: 6686
It depends. Often it's faster just to keep the desired rows rather than delete all the undesired. That is, copy the good rows to another table, trunc the table, copy the good rows back in.

Otherwise you can DELETE in batches as Sean suggests ... but don't forget the filter on the DELETE statement itself :-).

SQL DBA,SQL Server MVP('07, '08, '09)

Prosecutor James Blackburn, in closing argument in the "Fatal Vision" murders trial: "If in the future, you should cry a tear, cry one for them [the murder victims]. If in the future, you should say a prayer, say one for them. And if in the future, you should light a candle, light one for them."
CapnHector
CapnHector
SSC Eights!
SSC Eights! (917 reputation)SSC Eights! (917 reputation)SSC Eights! (917 reputation)SSC Eights! (917 reputation)SSC Eights! (917 reputation)SSC Eights! (917 reputation)SSC Eights! (917 reputation)SSC Eights! (917 reputation)

Group: General Forum Members
Points: 917 Visits: 1789
ScottPletcher (11/8/2012)
It depends. Often it's faster just to keep the desired rows rather than delete all the undesired. That is, copy the good rows to another table, trunc the table, copy the good rows back in.

Otherwise you can DELETE in batches as Sean suggests ... but don't forget the filter on the DELETE statement itself :-).


If im only keeping a small number of rows i will insert the rows into a new table but instead of truncating and reinserting the rows i will rename the tables. this will have much less down time as i dont have to reinsert the records into the original table after the truncate. once the tables are renamed i can keep the old table around to make sure every thing is working fine then just drop the table. Personally i think this method has a little more redundancy as the data is never gone until im sure i have every thing i need.


For faster help in answering any problems Please read How to post data/code on a forum to get the best help - Jeff Moden for the best way to ask your question.

For performance Issues see how we like them posted here: How to Post Performance Problems - Gail Shaw

Need to Split some strings? Jeff Moden's DelimitedSplit8K
Jeff Moden's Cross tab and Pivots Part 1
Jeff Moden's Cross tab and Pivots Part 2
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search