Technical Article

Automate Range Right Partition Management with PowerShell

,

In our environment, we have several partitioned tables that share a single partition scheme (with a range right, date partition function). Our objective is to keep X months of data in the partitioned tables (where X is determined by environment). I wanted a simple way to split off the trailing month of data from each partitioned table, merge the trailing boundary and then clean up the (now empty) file groups.

This Powershell script finds the tables associated with the Partition Scheme, creates copies of the tables, splits data from the targeted partition boundary into table copies, then (optionally) drops the stage tables, merges the partition boundary, and drops the associated file group.

This was my first realy foray into PowerShell. Some of the concepts were difficult to get my head around, but once I had the basics, it was actually pretty straightforward.

A huge chunk of this was adapted from Stuart Ozer's PartitionManager C# class library. The only reason I decided not to use that particular implementation was the requirement to install components from SQL 2008 (we are still lagging behind on SQL 2005). Also, I wanted to be able to manage multiple tables for one partition scheme without calling the script several times.

This has only been tested on SQL 2005 (not SQL 2008).

 

The Partition Function is defined as:

/****** Object: PartitionFunction [pf_FACT_DATA_DATE] Script Date: 12/17/2009 12:23:54 ******/
CREATE PARTITION FUNCTION [pf_FACT_DATA_DATE](datetime) AS RANGE RIGHT FOR
VALUES (
N'2007-04-01T00:00:00', N'2007-05-01T00:00:00', N'2007-06-01T00:00:00', N'2007-07-01T00:00:00', N'2007-08-01T00:00:00', N'2007-09-01T00:00:00', N'2007-10-01T00:00:00', N'2007-11-01T00:00:00', N'2007-12-01T00:00:00',
N'2008-01-01T00:00:00', N'2008-02-01T00:00:00', N'2008-03-01T00:00:00', N'2008-04-01T00:00:00', N'2008-05-01T00:00:00', N'2008-06-01T00:00:00', N'2008-07-01T00:00:00', N'2008-08-01T00:00:00', N'2008-09-01T00:00:00', N'2008-10-01T00:00:00', N'2008-11-01T00:00:00', N'2008-12-01T00:00:00',
N'2009-01-01T00:00:00', N'2009-02-01T00:00:00', N'2009-03-01T00:00:00', N'2009-04-01T00:00:00', N'2009-05-01T00:00:00', N'2009-06-01T00:00:00', N'2009-07-01T00:00:00', N'2009-08-01T00:00:00', N'2009-09-01T00:00:00', N'2009-10-01T00:00:00', N'2009-11-01T00:00:00', N'2009-12-01T00:00:00')

The Partition Scheme is define as:

/****** Object: PartitionScheme [ps_FACT_DATA_DATE] Script Date: 12/17/2009 12:24:53 ******/
CREATE PARTITION SCHEME [ps_FACT_DATA_DATE] AS PARTITION [pf_FACT_DATA_DATE] TO (
[FACT_EMPTY], [FACT_2007_M04], [FACT_2007_M05], [FACT_2007_M06], [FACT_2007_M07], [FACT_2007_M08], [FACT_2007_M09], [FACT_2007_M10], [FACT_2007_M11], [FACT_2007_M12],
[FACT_2008_M01], [FACT_2008_M02], [FACT_2008_M03], [FACT_2008_M04], [FACT_2008_M05], [FACT_2008_M06], [FACT_2008_M07], [FACT_2008_M08], [FACT_2008_M09], [FACT_2008_M10], [FACT_2008_M11], [FACT_2008_M12],
[FACT_2009_M01], [FACT_2009_M02], [FACT_2009_M03], [FACT_2009_M04], [FACT_2009_M05], [FACT_2009_M06], [FACT_2009_M07], [FACT_2009_M08], [FACT_2009_M09], [FACT_2009_M10], [FACT_2009_M11], [FACT_2009_M12])

###############################################################################
# File:        SwitchPartition.ps1
# Author:     Hugh Scott
# Date:        2009/12/03
# 
# Description:
# Script switches data from a partitioned table to a staging table in preparation
# for dropping the staging table. Script identifies all tables associated to 
# a particular PartitionScheme, builds the necessary staging table, switches the
# partition and then (optionally) drops the staging table(s).
#
# ToDo:
# 1. Add a switch to allow the user to optionally specify whether or not to perform
# a merge (remove that partition from the partition scheme)
# 2. Convert $partitionNumber to $boundaryValue
#
# Usage:
#
# .\SwitchPartition.ps1 Server Database stageTableSuffix partitionSchemeName boundaryDate [-keepStageTable] [-keepPartition] [-keepFileGroup]
#
###############################################################################
param (
    [string]$serverName ,         
    [string]$databaseName ,     
    [string]$stageTableSuffix ,     
    [string]$partitionSchemeName ,
    [datetime]$boundaryDate ,
    [switch]$keepStageTable ,
    [switch]$keepPartition ,
    [switch]$keepFileGroup
)

## Logical check of Parameters:
## 1. If $keepFileGroup is empty, then $keepStageTable and $keepPartition must also be empty
## 2. If $keepPartition is empty, then $keepStageTable must also be empty
## 3. Validate $server
## 4. Validate $databaseName
## 5. Validate $partitionSchemeName
if (!$keepFileGroup -and ($keepStageTable -or $keepPartition))
{
    Throw("Error: If file group is to be dropped, stage table(s) must be dropped and partition merged.")
}
if (!$keepPartition -and $keepStageTable)
{
    Throw("Error: If partition is to be merged, stage table(s) must be dropped.")
}

## Create a scripting object, specify options and then execute:
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | Out-Null
$srv = New-Object ("Microsoft.SqlServer.Management.SMO.Server")($serverName)
$db = $srv.Databases[$databaseName]
if($db.Name -eq $Null)
{
    Throw("Error: Unable to connect to database $databaseName on server $serverName.")
}

## Validate that $partitionSchemeName exists
$ps = $db.PartitionSchemes[$partitionSchemeName]
if($ps.Name -eq $Null)
{
    Throw("Error: Partition Scheme $partitionSchemeName does not exist")
}

## Determine partitionNumber from boundaryDate, spin through the rangeValues
## if someone has a better method, please let me know!
$pf = $db.PartitionFunctions[$ps.PartitionFunction]
$rangeValues = $pf.RangeValues
$i = 1
foreach($Value in $rangeValues)
{
    if($Value -eq $boundaryDate)
    {
        $partitionNumber = $i
    }
    $i = $i + 1
}

## Validate the resulting $partitionNumber value
## The resulting $partitionNumber can't be null
if($partitionNumber -eq $Null)
{
    Throw ("Error: Invalid boundary date $boundaryDate!")
}
else
{
    # If this is a range-right scheme, we have to increment the partition number by 1
    # At least, I THINK this is why we have to increment it by one
    if ($pf.RangeType -eq "Right")
    {
        $partitionNumber = $partitionNumber + 1
    }
    
    # $partitionNumber must be > 1 and < max number of partitions
    if($partitionNumber -le 1 -or $partitionNumber -ge $pf.NumberOfPartitions )
    {
    Throw ("Error: Invalid boundary date $boundaryDate!")
    }
}

## Create a collection of all tables associated to the PartitionScheme
## 1. Iterate through the table collection, 
## 2. Create a copy of the table,
## 3. Switch the data from the specified partition into the table 
## 4. Drop the copy of the table
$partitionTables = $db.tables | Where-Object {$_.PartitionScheme -eq $partitionSchemeName}
foreach ($partitionTable in $partitionTables)
{
    $stageSchemaName = $partitionTable.Schema
    $stageTableName = $partitionTable.Name + $stageTableSuffix
    # Write-Host $stageTableName

    # Test for the existence of the staging table, if it exists, drop it and then create it
    $testTable = $db.tables | Where-Object {$_.schema -eq $stageSchemaName-and$_.name -eq $stageTableName}
    If($testTable -ne $Null)
    {
        $testTable.Drop()
    }
    # Create the staging table
    $stageTable = New-Object("Microsoft.SqlServer.Management.SMO.Table")($db, $stageTableName, $stageSchemaName)

    # Create a copy of the partitioned table
    foreach    ($Col in $partitionTable.Columns)
    {
        $stageCol = New-Object("Microsoft.SqlServer.Management.SMO.Column")($stageTable, $Col.name, $Col.datatype)
        $stageCol.Collation = $Col.Collation
        $stageCol.Nullable = $Col.Nullable
        $stageCol.ComputedText = $Col.ComputedText
        $stageCol.Computed = $Col.Computed
        $stageCol.Default = $Col.Default
    #    if($Col.DefaultConstraint -ne $null)
    #    {
    #        $stageCol.AddDefaultConstraint($stageTable.name + "_" + $Col.DefaultConstraint.Name)
    #        $stageCol.DefaultConstraint.Text = $Col.DefaultConstraint.Text        
    #    }
        $stageCol.IsPersisted = $Col.IsPersisted
        $stageCol.DefaultSchema = $Col.DefaultSchema
        $stageCol.RowGuidCol = $Col.RowGuidCol
        
        $stageTable.Columns.Add($stageCol)
    }
    
    $stageTable.AnsiNullsStatus = $partitionTable.AnsiNullsStatus
    $stageTable.QuotedIdentifierStatus = $partitionTable.QuotedIdentifierStatus
    $stageTable.TextFileGroup = $partitionTable.TextFileGroup
    $stageTable.FileGroup = $db.PartitionSchemes[$partitionTable.PartitionScheme].FileGroups[$partitionNumber - 1]
    
    $stageTable.Create()

    # If it has one, create the clustered index on the table
    if($partitionTable.HasClusteredIndex)
    {
        foreach ($index in $partitionTable.Indexes)
        {
         if ($index.IsClustered)
         {
             $stageIndexName = $stageTable.Name + "_" + $index.Name
                $stgIndex = New-Object("Microsoft.SqlServer.Management.SMO.Index")($stageTable, $stageIndexName)
                foreach ($iCol in $index.IndexedColumns)
                {
                 $stgICol = New-Object("Microsoft.SqlServer.Management.SMO.IndexedColumn")($stgIndex, $iCol.Name, $iCol.Descending)
                 $stgICol.IsIncluded = $iCol.IsIncluded
                 $stgIndex.IndexedColumns.Add($stgICol)
                }
                $stgIndex.IndexKeyType = $index.IndexKeyType
                $stgIndex.IsClustered = $index.IsClustered
                $stgIndex.IsUnique = $index.IsUnique
                $stgIndex.CompactLargeObjects = $index.CompactLargeObjects
                $stgIndex.IgnoreDuplicateKeys = $index.IgnoreDuplicateKeys
                $stgIndex.IsFullTextKey = $index.IsFullTextKey
                $stgIndex.PadIndex = $index.PadIndex
                $stgIndex.FileGroup = $db.PartitionSchemes[$index.PartitionScheme].FileGroups[$partitionNumber - 1]
                
                ## add the partitioning column to the index if it is not already there
                $partitionKeyName = $index.PartitionSchemeParameters[0].Name
                if ($stgIndex.IndexedColumns[$partitionKeyName] -eq $Null)
                {
                 $stgICol = New-Object("Microsoft.SqlServer.Management.SMO.IndexedColumn")($stgIndex, $partitionKeyName)
                 ## It is added as a Key to the Clustered index and as an Include column to a Nonclustered
                 $stgICol.IsIncluded = !$stgIndex.IsClustered
                 $stgIndex.IndexedColumns.Add($stgICol)
                }
                
                $stgIndex.Create()
         # break
         }
        }        
    }
    
    # Switch partition (put the data in the indicated partition into the staging table
    $partitionTable.SwitchPartition($partitionNumber, $stageTable)
    
    # If keepTable is specified, then keep the staging table (for testing/validation
    # Otherwise, drop it
    if(!$keepStageTable)
    {
        $stageTable.Drop()
    }    
    else
    {
        Write-Host "Task not performed: Drop table $stageTable.ToString()"
    }
}

## Alter partition function; merge the specified boundary
if(!$keepPartition)
{
    $pf.MergeRangePartition($boundaryDate)
}
else
{
    Write-Host "Task not performed: Merge Range Partition $boundaryDate"
}

## If specified, drop filegroup from the database 
if(!$keepFileGroup)
{
    $fileGroupName = $db.PartitionSchemes[$partitionSchemeName].FileGroups[$partitionNumber - 1]
    $fileGroup = $db.FileGroups[$fileGroupName]
    $fileGroup.Drop()
}
else
{
    $fileGroup = $db.PartitionSchemes[$partitionSchemeName].FileGroups[$partitionNumber - 1]
    Write-Host "Task not performed: Drop filegroup name $fileGroup.Name"
}

Rate

5 (2)

You rated this post out of 5. Change rating

Share

Share

Rate

5 (2)

You rated this post out of 5. Change rating