speed up deleting million of records from a table with one filter

  • hi all,

    I want to delete million of records from a table with only one filter of varchar(1) column

    for that which one is best option.....

  • What do you mean with filter?

    Something like this?

    DELETE FROM myTable WHERE columnA = 'B'

    ps: why define a VARCHAR(1) column? What is wrong with CHAR(1). If you do have millions of rows, you're literally throwing storage away!

    Need an answer? No, you need a question
    My blog at https://sqlkover.com.
    MCSE Business Intelligence - Microsoft Data Platform MVP

  • hi

    It's actually char(1) column only.I am not asking abt that.

    I am asking batch processing is better or any other option will be better.

    Because I am going to delete million of records it's time consuming process.so that I am asking...

  • kuppurajdpm (11/8/2012)


    hi

    It's actually char(1) column only.I am not asking abt that.

    I am asking batch processing is better or any other option will be better.

    Because I am going to delete million of records it's time consuming process.so that I am asking...

    And I asked something about the filter. It's important that we know exactly what you're trying to achieve.

    Need an answer? No, you need a question
    My blog at https://sqlkover.com.
    MCSE Business Intelligence - Microsoft Data Platform MVP

  • Simplest solution is to use a WITH. e.g.

    WITH results ( <select> ) DELETE FROM results –

  • If you want to prevent locking your table for an extended period of time you can do this in batches.

    --This will ensure that the value of @@ROWCOUNT for the loop will be > 0

    select 1

    while @@ROWCOUNT > 0

    begin

    with DeleteList as (select top 1000 columnA from MyTable order by [Clustered Index])

    --order by clustered index so the engine should not have to revisit any given page more than neccesary.

    delete DeleteList

    end

    _______________________________________________________________

    Need help? Help us help you.

    Read the article at http://www.sqlservercentral.com/articles/Best+Practices/61537/ for best practices on asking questions.

    Need to split a string? Try Jeff Modens splitter http://www.sqlservercentral.com/articles/Tally+Table/72993/.

    Cross Tabs and Pivots, Part 1 – Converting Rows to Columns - http://www.sqlservercentral.com/articles/T-SQL/63681/
    Cross Tabs and Pivots, Part 2 - Dynamic Cross Tabs - http://www.sqlservercentral.com/articles/Crosstab/65048/
    Understanding and Using APPLY (Part 1) - http://www.sqlservercentral.com/articles/APPLY/69953/
    Understanding and Using APPLY (Part 2) - http://www.sqlservercentral.com/articles/APPLY/69954/

  • It depends. Often it's faster just to keep the desired rows rather than delete all the undesired. That is, copy the good rows to another table, trunc the table, copy the good rows back in.

    Otherwise you can DELETE in batches as Sean suggests ... but don't forget the filter on the DELETE statement itself :-).

    SQL DBA,SQL Server MVP(07, 08, 09) A socialist is someone who will give you the shirt off *someone else's* back.

  • ScottPletcher (11/8/2012)


    It depends. Often it's faster just to keep the desired rows rather than delete all the undesired. That is, copy the good rows to another table, trunc the table, copy the good rows back in.

    Otherwise you can DELETE in batches as Sean suggests ... but don't forget the filter on the DELETE statement itself :-).

    If im only keeping a small number of rows i will insert the rows into a new table but instead of truncating and reinserting the rows i will rename the tables. this will have much less down time as i dont have to reinsert the records into the original table after the truncate. once the tables are renamed i can keep the old table around to make sure every thing is working fine then just drop the table. Personally i think this method has a little more redundancy as the data is never gone until im sure i have every thing i need.


    For faster help in answering any problems Please read How to post data/code on a forum to get the best help - Jeff Moden[/url] for the best way to ask your question.

    For performance Issues see how we like them posted here: How to Post Performance Problems - Gail Shaw[/url]

    Need to Split some strings? Jeff Moden's DelimitedSplit8K[/url]
    Jeff Moden's Cross tab and Pivots Part 1[/url]
    Jeff Moden's Cross tab and Pivots Part 2[/url]

Viewing 8 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic. Login to reply